-- Logs begin at Mon 2022-10-24 13:00:37 UTC, end at Mon 2022-10-24 14:05:31 UTC. -- Oct 24 13:28:08.370888 dv-cinder-862490-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 24 13:28:08.378241 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 24 13:28:08 2022] *** Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: nodename: dv-cinder-862490-2-iscsi Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: machine: x86_64 Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: clock source: unix Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: pcre jit disabled Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: detected number of CPU cores: 4 Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: current working directory: / Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: detected binary path: /usr/bin/uwsgi-core Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: your processes number limit is 47680 Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: your memory page size is 4096 bytes Oct 24 13:28:08.380208 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: detected max file descriptor number: 2048 Oct 24 13:28:08.380996 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: lock engine: pthread robust mutexes Oct 24 13:28:08.380996 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: thunder lock: enabled Oct 24 13:28:08.380996 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 24 13:28:08.380996 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 24 13:28:08.421521 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: Python main interpreter initialized at 0x555fb1da59c0 Oct 24 13:28:08.421521 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: python threads support enabled Oct 24 13:28:08.421521 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: your server socket listen backlog is limited to 100 connections Oct 24 13:28:08.421521 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: your mercy for graceful operations on workers is 90 seconds Oct 24 13:28:08.421890 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: mapped 403077 bytes (393 KB) for 2 cores Oct 24 13:28:08.421990 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: *** Operational MODE: preforking *** Oct 24 13:28:08.422283 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: *** uWSGI is running in multiple interpreter mode *** Oct 24 13:28:08.422283 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: spawned uWSGI master process (pid: 81675) Oct 24 13:28:08.422482 dv-cinder-862490-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 24 13:28:08.423300 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: spawned uWSGI worker 1 (pid: 81687, cores: 1) Oct 24 13:28:08.423400 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: spawned uWSGI worker 2 (pid: 81688, cores: 1) Oct 24 13:28:08.425351 dv-cinder-862490-2-iscsi devstack@keystone.service[81675]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 24 13:28:12.675658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 13:28:12.675658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: from cryptography.utils import int_from_bytes Oct 24 13:28:12.676430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 13:28:12.676430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: from cryptography.utils import int_from_bytes Oct 24 13:28:12.676847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 13:28:12.676847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: from cryptography.utils import int_from_bytes Oct 24 13:28:12.677324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 13:28:12.677324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: from cryptography.utils import int_from_bytes Oct 24 13:28:12.725892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 13:28:12.726064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 13:28:12.726277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] command line args: [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 13:28:12.726585 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 13:28:12.727352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ================================================================================ {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 13:28:12.727608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] admin_token = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.727849 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] config_dir = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.728026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.728113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 13:28:12.728187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] config_source = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.728261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 13:28:12.728367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] debug = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.728454 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] command line args: [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 13:28:12.728562 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.728868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 13:28:12.728868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ================================================================================ {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 13:28:12.729061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.729172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] admin_token = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.729265 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.729361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] config_dir = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.729452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.729541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.729638 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.729742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] config_source = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.729835 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.729925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] debug = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.730021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_config_append = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.730123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.730338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.730441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.730546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_dir = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.730638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.730728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_file = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.730825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.730933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.731018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.731134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.731206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.731324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.731403 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_config_append = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.731526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.731603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.731714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.731794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_dir = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.731907 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.731989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_file = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.732101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.732173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.732283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.732362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.732479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.732560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.732637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.732754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.732829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.732941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.732941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.733138 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.733215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.733328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.733406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.733482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.733629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.733725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.733862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.734093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.734189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.734353 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.734482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] publish_errors = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.734592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.734668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.734820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.734936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.735055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.735166 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.735282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.735440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.735553 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.735672 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] standard_threads = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.735788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] publish_errors = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.735899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.736026 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.736147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.736258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.736373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.736485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.736595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] use_journal = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.736712 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.736824 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] use_json = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.736824 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] use_stderr = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.737038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.737149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] use_syslog = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.737258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] standard_threads = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.737419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.737539 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.737651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.737767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.737878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.737994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.737994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] use_journal = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.738239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.738362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] use_json = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.738476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.738593 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] use_stderr = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.738593 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] use_syslog = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.738818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.738929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:28:12.739046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.739163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.739281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.739392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.739512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.739624 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.739733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.739844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.739959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.740070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.740192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.740311 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.740428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.740539 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.740649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.740759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.740880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.740880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.741094 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.741204 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.741324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.741435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.741558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.741558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.741778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.741889 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.741999 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.742116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.742301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.742490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.742680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.742841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.742964 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.743083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.743278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.743401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.743518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.743629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.743739 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.743739 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.743955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.744065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.744174 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.744289 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.744400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.744520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.744637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.744748 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.744864 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.744975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.745091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.745211 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.745322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.745438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.745550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.745666 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.745777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.745894 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.746005 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.746121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.746279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.746399 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.746511 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.746630 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.746741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.746835 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.746933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.747050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.747161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.747279 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.747391 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.747507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.747619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.747728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.747838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.747955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.748066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.748176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.748286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.748396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.748505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.748677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.748789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.748899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.749010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.749120 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.749230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.749347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.749458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.749568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.749679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.749796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.749907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.750025 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.750135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.750262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.750373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.750483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.750576 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.750682 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.750778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.750875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.750974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.751065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.751155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.751254 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.751347 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.751438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.751562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.751683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.751822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.751943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.752054 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.752170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.752289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.752407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.752525 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.752645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.752747 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.752864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.752982 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.753093 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.753212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.753325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.753451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.753569 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.753686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.753803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.753916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.754028 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.754136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.754294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.754405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.754506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.754615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.754725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.754847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.754963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.755075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.755194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.755307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.755418 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.755537 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.755650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.755761 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.755872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.755985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.756096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.756216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.756328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.756439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.756557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.756669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.756780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.756903 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.757016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.757127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.757238 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.757349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.757451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.757557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.757668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.757794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.757935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.758126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.758326 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.758489 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.758606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.758718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.758836 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.758949 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.759060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.759181 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.759295 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.759406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.759524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.759524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.759735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.759921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.760035 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.760147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.760259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.760377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.760505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.760627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.760748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.760861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.760973 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.761085 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.761196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.761308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.761427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.761541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.761652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.761763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.761874 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.761986 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.762105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.762232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.762346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.762458 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.762569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.762680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.external = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.762807 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.762922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.763034 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.763156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.763271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.763409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.763535 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.763655 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.password = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.763776 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.763890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] auth.token = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.764003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.764123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.764123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.764340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.764340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.external = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.764561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.764674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.764785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.764785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.764994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.765115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.765229 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.765341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.765454 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.password = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.765566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.765678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] auth.token = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.765798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.765911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.766024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.766142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.766283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.766395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.766395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.766615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.766727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.766839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.766839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.767054 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.767054 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.767275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.767386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.767499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.767611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.767723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.767844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.767964 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.768078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.768190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.768308 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.768421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.768541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.768655 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.768767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.768879 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.768999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.769113 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.769232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.769346 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.769458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.769571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.769684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.769796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.769916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.770036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.770036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.770264 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.770377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.770490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.770614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.770731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.770843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.770955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.771068 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.771188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.771302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.771414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.771526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.771638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.771760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.771889 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.772009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.772126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.772240 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.772352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.772464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.772588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.772710 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.772823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.772936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.773048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.773166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.773349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.773981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.774132 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.774323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.774500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.774697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.774876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.775002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.775117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.775231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.775343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.775456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.775577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.775698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.775812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.775933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.776054 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.776168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.776290 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.776404 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.776517 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.776620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.776741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.776855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.776978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.777092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.777205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.777318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.777430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.777543 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.777663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.777778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.777890 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.778004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.778116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.778283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.778413 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.778529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.778642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.778756 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.778869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.778982 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.779109 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.779224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.779337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.779450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.779563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.779675 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.779796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.779911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.780024 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.780138 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.780251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.780364 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.780485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.780600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.780713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.780826 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.780938 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.781051 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.781173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.781288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.781401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.781515 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.781627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.781749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.781864 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.781977 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.782089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.782217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.782331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.782456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.782579 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.782693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.782806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.782919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.783031 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.783152 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.783268 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.783381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.783493 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.783606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.783724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.783845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.783964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.784076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.784189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.784303 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.784416 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.784540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.784654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.784767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.784879 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.784992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.785104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.785264 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.785382 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.785495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.785607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.785721 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.785833 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.785955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.786069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.786247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.786363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.786477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.786590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.786715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.786831 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.786944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.787093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.787209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.787209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.787428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.787542 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.787652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.787763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.787909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.788024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.788145 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.788259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.788371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.788485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.788597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.788709 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.788830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.788944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.789182 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.789874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.790071 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.790221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.790351 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.790474 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.790590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.790703 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.790816 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.790928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.791049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.791163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.791275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.791388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.791500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.791613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.791734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.791848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.792218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.792331 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.792443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.792555 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.792676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.792676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.792889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.793001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.793113 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.793225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.793346 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.793460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.793572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.793684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.793798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.793913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.794034 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.794147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.794309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.794422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.794534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.794646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.794772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.794888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.795000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.795113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.795225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.795337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.795458 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.795573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.795685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.795797 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.795917 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.796031 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.796153 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.796267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.796379 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.796492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.796605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.796718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.796838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] role.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.796953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.797066 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] role.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.797179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.797291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.797476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.797593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.797705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.797817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.797932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.798044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.798182 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.798298 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.798412 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.798524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.798637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.798749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.798873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] role.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.798990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.799102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] role.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.799215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.799327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.799440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.799561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.799675 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.799794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.799908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.800023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.800136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.800256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.800371 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.800484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.800597 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.800715 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.800829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.800950 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.801065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.801178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.801291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.801403 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.801516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.801636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.801750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.801869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.801984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.802097 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.802248 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.802375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.802491 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.802603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.802715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.802828 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.802949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.802949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.803163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.803276 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.803388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.803500 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.803620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.803734 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.803846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.803962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.804074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.804192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.804315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.804428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.804540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.804700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.804871 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.805043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.805236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.805431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.805602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.805723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.805837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.805950 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.806072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.806202 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.806319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.806433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.806545 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.806658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.806788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.806905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.807018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.807131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.807244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.807359 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.807480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.807594 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.807707 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.807820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.807923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.807923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.808141 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.808255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.808368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.808482 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.808482 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.808692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.808812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.808926 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.809048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.809170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.809284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.809397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.809511 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.809631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.connection = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.809746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.809859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.809972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.810084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.810215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.810341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.810456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.810571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.810684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.810806 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.810929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.811043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.811155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.811318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.811318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.811533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.connection = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.811646 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.811758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.811870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.811991 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.812106 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.812219 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.812332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.812444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.812556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.812705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.812822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.812934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.813046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.813158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.813271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.813392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.813506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.813618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.813731 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.813868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.813989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81687) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 13:28:12.814110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.814110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.814110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.814110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.814110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.814110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.814110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.814110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:28:12.814110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81688) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 13:28:12.840543 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.840768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.840936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.841022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.841169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.841396 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.849211 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.849589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.877766 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.877992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.878508 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.878737 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.930455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.931375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.978545 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.978774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.979632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:12.979866 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.131783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.131967 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.132109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.132191 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.132267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.132363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.132441 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.132557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.132641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.132807 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.132899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.133030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.156195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.156304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.156466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.156561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.156697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.156780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.156919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.157002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.157138 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.157230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.157350 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.157431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.157560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.157642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.169389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.169648 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.182821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.182996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.183129 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.183224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.183363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.183446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.183587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.183677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.183808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.183900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.184014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81687) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.184095 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.184228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.184443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81688) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:28:13.200829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.201036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 13:28:13.201163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.201373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.201455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 13:28:13.201572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 13:28:13.201709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.201842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.201925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 13:28:13.202100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.202205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.202439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.202647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.202728 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.202907 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.202987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.212313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.212519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.212605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.212801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.212884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.213072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.213158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.213353 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.220023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.220335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.222909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.223194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.223276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.223487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.223575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.223761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.223856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.224042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.224125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.224302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.224385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.224570 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.224652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.224834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.224914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.225106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.225195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.225377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.225458 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.225649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.225730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.225915 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.249116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.249707 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.250590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.250876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.251212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.251499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.256161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.256449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.256726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.257208 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.257497 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.257813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.263259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.263548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.264276 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.264560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.268705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.268997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.269269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.269351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.269554 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.269646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.269822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.269904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.270090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.270197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.270391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.270475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.270667 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.270748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.270945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.271214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.286110 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.286413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.288407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.288694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.290776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.293310 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.294235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.294531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.296562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.296861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.298719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.301090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.301899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.304318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.305696 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.305990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.306303 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.306594 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.306876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.307152 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.308192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.308492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.308778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.309081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.309376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.309666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.322815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.323100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.323365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.323623 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.325513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.325801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.326068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.326360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.330424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.331956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.333185 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.333483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.333767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.334756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.336480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.336567 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.336833 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.336920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.337222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.337503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.337784 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.339790 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.340085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.340366 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.340647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.341032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.348834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.349121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.349509 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.349789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.350076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.350383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.350668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.352849 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.353139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.353421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.353701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.354091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.354394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.354684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.366527 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.370107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.370554 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.372976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.373457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.374492 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.376086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.376378 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.376457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.376654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.377877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.379394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.379680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.380089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.383005 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.384473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.386572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.387464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.387751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.388029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.388113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.388313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.390917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.391315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.391602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.391882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.396037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.396328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.396607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.399949 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.400241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.400521 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.404386 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81687) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.404671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.404949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.405219 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.405484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.405750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.406020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.406327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.406598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.406872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.407140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.407391 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81688) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:28:13.407484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.407676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.407768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.407948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81687) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.408038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.408223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.408497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.408770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.409044 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.409333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.409610 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.409877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.410153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.410450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.410722 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.410998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81688) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:28:13.442372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555fb1da59c0 pid: 81687 (default app) Oct 24 13:28:13.445651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd4153-ca4f-4c4f-9bed-76f23cbf63b0 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:13.445900 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555fb1da59c0 pid: 81688 (default app) Oct 24 13:28:13.446061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd4153-ca4f-4c4f-9bed-76f23cbf63b0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:13.446361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd4153-ca4f-4c4f-9bed-76f23cbf63b0 None None] PATH_INFO: `/v3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:13.450473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Mon Oct 24 13:28:13 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:20.193357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb818e39-97cc-4ff9-a770-2b7880558f81 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:20.193860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb818e39-97cc-4ff9-a770-2b7880558f81 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:20.193965 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb818e39-97cc-4ff9-a770-2b7880558f81 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:20.199248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:20.204647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f44e01-36d8-45d7-9f4e-a9377777e889 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:20.205107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f44e01-36d8-45d7-9f4e-a9377777e889 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:20.205347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f44e01-36d8-45d7-9f4e-a9377777e889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:20.275701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: INFO dbcounter [None req-b6f44e01-36d8-45d7-9f4e-a9377777e889 None None] Registered counter for database keystone Oct 24 13:28:20.295317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG oslo_db.sqlalchemy.engines [None req-b6f44e01-36d8-45d7-9f4e-a9377777e889 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81687) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 13:28:20.301456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writer thread running {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 13:28:20.479440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG passlib.handlers.bcrypt [None req-b6f44e01-36d8-45d7-9f4e-a9377777e889 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81687) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 24 13:28:20.479739 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG passlib.handlers.bcrypt [None req-b6f44e01-36d8-45d7-9f4e-a9377777e889 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81687) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 24 13:28:20.537906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b6f44e01-36d8-45d7-9f4e-a9377777e889 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:20.539558 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b6f44e01-36d8-45d7-9f4e-a9377777e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:20.667126 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 13:28:21.117946 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb818e39-97cc-4ff9-a770-2b7880558f81 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:21.190376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: INFO dbcounter [None req-fb818e39-97cc-4ff9-a770-2b7880558f81 None None] Registered counter for database keystone Oct 24 13:28:21.212503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG oslo_db.sqlalchemy.engines [None req-fb818e39-97cc-4ff9-a770-2b7880558f81 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81688) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 13:28:21.218778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writer thread running {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 13:28:21.339507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb818e39-97cc-4ff9-a770-2b7880558f81 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:21.343821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081f70e-8041-4c58-bd5d-9eaaad0bd8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:21.344442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f70e-8041-4c58-bd5d-9eaaad0bd8bf admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:21.344723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f70e-8041-4c58-bd5d-9eaaad0bd8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:21.344991 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f70e-8041-4c58-bd5d-9eaaad0bd8bf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:21.353829 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081f70e-8041-4c58-bd5d-9eaaad0bd8bf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:21.404932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5081f70e-8041-4c58-bd5d-9eaaad0bd8bf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 24 13:28:21.405215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081f70e-8041-4c58-bd5d-9eaaad0bd8bf admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:21.415267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Oct 24 13:28:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:28:21.421445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f44e01-36d8-45d7-9f4e-a9377777e889 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:21.438771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f44e01-36d8-45d7-9f4e-a9377777e889 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:21.443193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296c95f-9293-43f2-9070-406f99612fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:21.444305 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296c95f-9293-43f2-9070-406f99612fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:21.444586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296c95f-9293-43f2-9070-406f99612fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:21.444858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296c95f-9293-43f2-9070-406f99612fa3 admin admin] PATH_INFO: `/v3/projects/73bbcd87686646c2a2c2fdd8f0e7bdd8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:21.448709 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296c95f-9293-43f2-9070-406f99612fa3 admin admin] RBAC: Authorizing `identity:get_project(project_id=73bbcd87686646c2a2c2fdd8f0e7bdd8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:21.500296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7296c95f-9293-43f2-9070-406f99612fa3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 24 13:28:21.500643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296c95f-9293-43f2-9070-406f99612fa3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:21.505157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Oct 24 13:28:21 2022] GET /identity/v3/projects/73bbcd87686646c2a2c2fdd8f0e7bdd8 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:23.657143 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064eb2e-1060-4347-ad22-6da3f0ccc5b5 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:23.657685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064eb2e-1060-4347-ad22-6da3f0ccc5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:23.657685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064eb2e-1060-4347-ad22-6da3f0ccc5b5 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:23.661763 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:23.667866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d48f94-965e-4af1-b30e-dd640b864e20 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:23.668250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d48f94-965e-4af1-b30e-dd640b864e20 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:23.668416 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d48f94-965e-4af1-b30e-dd640b864e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:23.688500 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-16d48f94-965e-4af1-b30e-dd640b864e20 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:23.690005 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-16d48f94-965e-4af1-b30e-dd640b864e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:23.698236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 13:28:23.967190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064eb2e-1060-4347-ad22-6da3f0ccc5b5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:23.979309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064eb2e-1060-4347-ad22-6da3f0ccc5b5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:23.983732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d84af-08b3-439e-8603-68e910805c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:23.984325 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d84af-08b3-439e-8603-68e910805c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:23.984605 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d84af-08b3-439e-8603-68e910805c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:23.984874 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d84af-08b3-439e-8603-68e910805c08 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:23.993255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d84af-08b3-439e-8603-68e910805c08 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:23.995043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d84af-08b3-439e-8603-68e910805c08 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:24.003601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Oct 24 13:28:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:28:24.009411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d48f94-965e-4af1-b30e-dd640b864e20 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:24.018314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d48f94-965e-4af1-b30e-dd640b864e20 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:24.022750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9836d22-805d-445b-85b6-144092d91e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:24.023501 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9836d22-805d-445b-85b6-144092d91e56 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:24.023779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9836d22-805d-445b-85b6-144092d91e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:24.024045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9836d22-805d-445b-85b6-144092d91e56 admin admin] PATH_INFO: `/v3/users/75330c8a27ec4acf96914b6033d2cd61` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:24.025349 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9836d22-805d-445b-85b6-144092d91e56 admin admin] RBAC: Authorizing `identity:get_user(user_id=75330c8a27ec4acf96914b6033d2cd61)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:24.027196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9836d22-805d-445b-85b6-144092d91e56 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:24.028701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Mon Oct 24 13:28:24 2022] GET /identity/v3/users/75330c8a27ec4acf96914b6033d2cd61 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:26.199113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b867ff-f8b5-4891-a32c-7432894d9d8d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:26.199650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b867ff-f8b5-4891-a32c-7432894d9d8d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:26.199650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b867ff-f8b5-4891-a32c-7432894d9d8d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:26.201007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:26.207241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819719-2bc0-443a-98e4-01065032c3ba None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:26.207862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819719-2bc0-443a-98e4-01065032c3ba None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:26.207862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819719-2bc0-443a-98e4-01065032c3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:26.227368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-48819719-2bc0-443a-98e4-01065032c3ba None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:26.228573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-48819719-2bc0-443a-98e4-01065032c3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:26.313674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 6/11] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:28:26.330430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65703251-2afb-4666-b1f4-21b192f70a91 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:26.330717 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65703251-2afb-4666-b1f4-21b192f70a91 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:26.330992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65703251-2afb-4666-b1f4-21b192f70a91 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:26.332335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 6/12] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:26.337774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4fb0b-9cf5-4d09-b724-2a28409b6b64 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:26.338055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4fb0b-9cf5-4d09-b724-2a28409b6b64 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:26.338355 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4fb0b-9cf5-4d09-b724-2a28409b6b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:26.356445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-bef4fb0b-9cf5-4d09-b724-2a28409b6b64 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:26.357604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bef4fb0b-9cf5-4d09-b724-2a28409b6b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:26.374963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:26.429152 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:26.580887 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65703251-2afb-4666-b1f4-21b192f70a91 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:26.695114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4fb0b-9cf5-4d09-b724-2a28409b6b64 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:26.724731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65703251-2afb-4666-b1f4-21b192f70a91 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:26.758977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536df75-492c-40a1-a52a-bbdffd6635e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:26.759826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536df75-492c-40a1-a52a-bbdffd6635e5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:26.760106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536df75-492c-40a1-a52a-bbdffd6635e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:26.760374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536df75-492c-40a1-a52a-bbdffd6635e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:26.771464 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536df75-492c-40a1-a52a-bbdffd6635e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-9536df75-492c-40a1-a52a-bbdffd6635e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:26.782793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:26.783729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:26.784523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:28:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:26.790195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536df75-492c-40a1-a52a-bbdffd6635e5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:26.797165 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4fb0b-9cf5-4d09-b724-2a28409b6b64 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:26.824869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:26.831948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588797a-4960-4b20-82cb-3e6d95b8fab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:26.832507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588797a-4960-4b20-82cb-3e6d95b8fab6 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:26.832781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588797a-4960-4b20-82cb-3e6d95b8fab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:26.833049 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588797a-4960-4b20-82cb-3e6d95b8fab6 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:26.833718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588797a-4960-4b20-82cb-3e6d95b8fab6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:26.835122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588797a-4960-4b20-82cb-3e6d95b8fab6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:26.845078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.sql.core [None req-9588797a-4960-4b20-82cb-3e6d95b8fab6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:28:26.845078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:28:26.845078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [parameters: {'id': '51b1ce9c6ff44f84b0d425a4f892dc85', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 13:28:26.845078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81687) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-9588797a-4960-4b20-82cb-3e6d95b8fab6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:28:26.864190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result.read() Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 13:28:26.865149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application yield resource Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise self.value Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application session.commit() Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:28:26.866235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:28:26.867990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:28:26.869175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application result.read() Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:28:26.870071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [parameters: {'id': '51b1ce9c6ff44f84b0d425a4f892dc85', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 13:28:26.870997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:26.871871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 8/15] 10.222.0.20 () {60 vars in 1137 bytes} [Mon Oct 24 13:28:26 2022] POST /identity/v3/domains => generated 190 bytes in 171 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:28:26.872972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588797a-4960-4b20-82cb-3e6d95b8fab6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:26.890774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:26.894561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536df75-492c-40a1-a52a-bbdffd6635e5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:26.931575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7a74e-49f3-47b3-937d-44997e6385dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:26.932171 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7a74e-49f3-47b3-937d-44997e6385dc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:26.932452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7a74e-49f3-47b3-937d-44997e6385dc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:26.932724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7a74e-49f3-47b3-937d-44997e6385dc None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:26.933377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7a74e-49f3-47b3-937d-44997e6385dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:26.933672 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7a74e-49f3-47b3-937d-44997e6385dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:26.935079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7a74e-49f3-47b3-937d-44997e6385dc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:26.946482 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 8/16] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:26.951994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7a74e-49f3-47b3-937d-44997e6385dc None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:26.975465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588797a-4960-4b20-82cb-3e6d95b8fab6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:27.008878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18fe2-ead4-4f08-80b8-415e6b2068f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:27.009512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18fe2-ead4-4f08-80b8-415e6b2068f4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:27.009792 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18fe2-ead4-4f08-80b8-415e6b2068f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:27.010062 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18fe2-ead4-4f08-80b8-415e6b2068f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:27.019940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb18fe2-ead4-4f08-80b8-415e6b2068f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:27.021439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb18fe2-ead4-4f08-80b8-415e6b2068f4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-cbb18fe2-ead4-4f08-80b8-415e6b2068f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:28:27.037671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:27.039274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:28:27.040061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:28:27.040061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:28:27.040061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:27.040061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:28:27.044725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:27.045950 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18fe2-ead4-4f08-80b8-415e6b2068f4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:27.052003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7a74e-49f3-47b3-937d-44997e6385dc None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:27.084375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60157b-62d8-484d-8b35-d223b9a1a3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:27.084936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60157b-62d8-484d-8b35-d223b9a1a3ce None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:27.085214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60157b-62d8-484d-8b35-d223b9a1a3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:27.086083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60157b-62d8-484d-8b35-d223b9a1a3ce None admin] PATH_INFO: `/v3/users/75330c8a27ec4acf96914b6033d2cd61` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:27.087713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60157b-62d8-484d-8b35-d223b9a1a3ce None admin] RBAC: Authorizing `identity:get_user(user_id=75330c8a27ec4acf96914b6033d2cd61)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:27.089160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60157b-62d8-484d-8b35-d223b9a1a3ce None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:27.090765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:26 2022] GET /identity/v3/users/75330c8a27ec4acf96914b6033d2cd61 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:27.096383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60157b-62d8-484d-8b35-d223b9a1a3ce None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:27.114221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:27.146979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18fe2-ead4-4f08-80b8-415e6b2068f4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:27.181142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8307ec-579c-4034-b37f-576b7a5c414c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:27.181750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8307ec-579c-4034-b37f-576b7a5c414c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:27.182033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8307ec-579c-4034-b37f-576b7a5c414c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:27.182335 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8307ec-579c-4034-b37f-576b7a5c414c None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:27.182968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8307ec-579c-4034-b37f-576b7a5c414c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:27.183274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8307ec-579c-4034-b37f-576b7a5c414c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:27.184651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8307ec-579c-4034-b37f-576b7a5c414c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:27.196053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60157b-62d8-484d-8b35-d223b9a1a3ce None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:27.203813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:27.236125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f3485-7788-4281-910f-bc496fd27794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:27.236782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f3485-7788-4281-910f-bc496fd27794 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:27.237056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f3485-7788-4281-910f-bc496fd27794 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:27.237327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f3485-7788-4281-910f-bc496fd27794 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:27.238618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f3485-7788-4281-910f-bc496fd27794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:27.240099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f3485-7788-4281-910f-bc496fd27794 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:27.241747 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:27.248014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8307ec-579c-4034-b37f-576b7a5c414c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:27.359354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8307ec-579c-4034-b37f-576b7a5c414c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:27.389460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:27.398724 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f62c2-3a97-494a-971c-3bebc95555f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:27.399397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f62c2-3a97-494a-971c-3bebc95555f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:27.399671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f62c2-3a97-494a-971c-3bebc95555f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:27.399946 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f62c2-3a97-494a-971c-3bebc95555f3 None admin] PATH_INFO: `/v3/domains/default/users/75330c8a27ec4acf96914b6033d2cd61/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:27.401893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f62c2-3a97-494a-971c-3bebc95555f3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=75330c8a27ec4acf96914b6033d2cd61, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:27.403447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f62c2-3a97-494a-971c-3bebc95555f3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:27.517521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 11/21] 10.222.0.20 () {58 vars in 1361 bytes} [Mon Oct 24 13:28:27 2022] PUT /identity/v3/domains/default/users/75330c8a27ec4acf96914b6033d2cd61/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:28:29.717695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62484b6-c6d9-4c09-b5b8-e17d390f3f7d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:29.718254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62484b6-c6d9-4c09-b5b8-e17d390f3f7d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:29.718254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62484b6-c6d9-4c09-b5b8-e17d390f3f7d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:29.719889 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:29.725599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25590f32-033f-428d-840e-9db1ef1b2ae6 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:29.726056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25590f32-033f-428d-840e-9db1ef1b2ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:29.726353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25590f32-033f-428d-840e-9db1ef1b2ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:29.755011 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-25590f32-033f-428d-840e-9db1ef1b2ae6 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:29.757019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-25590f32-033f-428d-840e-9db1ef1b2ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:29.847145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:29.902427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4259d6e-e8e0-4dfb-a661-0a3a1e710ab3 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:29.902743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4259d6e-e8e0-4dfb-a661-0a3a1e710ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:29.902987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4259d6e-e8e0-4dfb-a661-0a3a1e710ab3 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:29.904593 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 12/24] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:29.909865 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b567ad5-f588-4248-bf6d-e7d3029ed36a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:29.910406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b567ad5-f588-4248-bf6d-e7d3029ed36a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:29.910897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b567ad5-f588-4248-bf6d-e7d3029ed36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:29.930157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3b567ad5-f588-4248-bf6d-e7d3029ed36a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:29.931725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3b567ad5-f588-4248-bf6d-e7d3029ed36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:30.016916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 13/25] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:28:30.245365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4259d6e-e8e0-4dfb-a661-0a3a1e710ab3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:30.280620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:30.356609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a5c6a-7c62-40e4-a7b0-66989c5d24e3 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:30.356927 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a5c6a-7c62-40e4-a7b0-66989c5d24e3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:30.357171 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a5c6a-7c62-40e4-a7b0-66989c5d24e3 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:30.361479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 14/26] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:30.368360 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7890db0-1f14-4964-a360-d5ba899f143d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:30.368896 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7890db0-1f14-4964-a360-d5ba899f143d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:30.369384 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7890db0-1f14-4964-a360-d5ba899f143d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:30.378220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4259d6e-e8e0-4dfb-a661-0a3a1e710ab3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:30.407206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a7890db0-1f14-4964-a360-d5ba899f143d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:30.408785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a7890db0-1f14-4964-a360-d5ba899f143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:30.442989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a5463-42a2-4475-af98-5d0bbf8a0985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:30.443654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a5463-42a2-4475-af98-5d0bbf8a0985 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:30.443939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a5463-42a2-4475-af98-5d0bbf8a0985 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:30.444215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a5463-42a2-4475-af98-5d0bbf8a0985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:30.476970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a5463-42a2-4475-af98-5d0bbf8a0985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:30.478706 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a5463-42a2-4475-af98-5d0bbf8a0985 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:30.499585 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-443a5463-42a2-4475-af98-5d0bbf8a0985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:28:30.515477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:30.516367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:28:30.517241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:28:30.517241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:28:30.517241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:30.521511 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 13/27] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:30.522809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a5463-42a2-4475-af98-5d0bbf8a0985 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:30.540909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 15/28] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:30.542609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7890db0-1f14-4964-a360-d5ba899f143d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:30.666891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7890db0-1f14-4964-a360-d5ba899f143d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:30.710047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:30.724827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc045afa-f660-4553-a92a-8da2e5723ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:30.725443 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc045afa-f660-4553-a92a-8da2e5723ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:30.726991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc045afa-f660-4553-a92a-8da2e5723ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:30.727563 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc045afa-f660-4553-a92a-8da2e5723ec0 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:30.727921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc045afa-f660-4553-a92a-8da2e5723ec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:30.728223 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc045afa-f660-4553-a92a-8da2e5723ec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:30.729722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc045afa-f660-4553-a92a-8da2e5723ec0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:30.730846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a5463-42a2-4475-af98-5d0bbf8a0985 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:30.755544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 16/29] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:30.756856 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c68e21-89f9-418f-8a05-d49a08a2bc39 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:30.757145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c68e21-89f9-418f-8a05-d49a08a2bc39 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:30.758399 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c68e21-89f9-418f-8a05-d49a08a2bc39 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:30.766090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 17/30] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:30.766957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c68e21-89f9-418f-8a05-d49a08a2bc39 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:30.770723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7319f-688a-4842-a23a-82bde6282189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:30.771571 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7319f-688a-4842-a23a-82bde6282189 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:30.772049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7319f-688a-4842-a23a-82bde6282189 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:30.772516 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7319f-688a-4842-a23a-82bde6282189 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:30.773618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7319f-688a-4842-a23a-82bde6282189 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:30.782193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7319f-688a-4842-a23a-82bde6282189 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:30.845054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:30.917096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c68e21-89f9-418f-8a05-d49a08a2bc39 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:30.953269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64226ea-e943-4b23-bbeb-5a52a5c89fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:30.954117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64226ea-e943-4b23-bbeb-5a52a5c89fae None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:30.954583 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64226ea-e943-4b23-bbeb-5a52a5c89fae None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:30.954994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64226ea-e943-4b23-bbeb-5a52a5c89fae None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:30.955775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64226ea-e943-4b23-bbeb-5a52a5c89fae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:30.957403 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64226ea-e943-4b23-bbeb-5a52a5c89fae None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:30.958212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 14/31] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Oct 24 13:28:30 2022] POST /identity/v3/roles => generated 219 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:30.959804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74ef3f-f376-4b00-9313-9cbc0bbd2404 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:30.960100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74ef3f-f376-4b00-9313-9cbc0bbd2404 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:30.960419 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74ef3f-f376-4b00-9313-9cbc0bbd2404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:30.996379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG passlib.handlers.bcrypt [None req-6f74ef3f-f376-4b00-9313-9cbc0bbd2404 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81688) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 24 13:28:30.997037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG passlib.handlers.bcrypt [None req-6f74ef3f-f376-4b00-9313-9cbc0bbd2404 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81688) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 24 13:28:31.017314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 18/32] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:28:30 2022] POST /identity/v3/projects => generated 301 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:31.018495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64226ea-e943-4b23-bbeb-5a52a5c89fae None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:31.087756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6f74ef3f-f376-4b00-9313-9cbc0bbd2404 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:31.090303 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6f74ef3f-f376-4b00-9313-9cbc0bbd2404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:31.232118 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:31.253449 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64226ea-e943-4b23-bbeb-5a52a5c89fae None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:31.305889 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 15/33] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:31.340960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888793a-9c7d-40ee-823e-ff691401bf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:31.341628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888793a-9c7d-40ee-823e-ff691401bf61 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:31.341909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888793a-9c7d-40ee-823e-ff691401bf61 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:31.342329 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888793a-9c7d-40ee-823e-ff691401bf61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:31.358420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888793a-9c7d-40ee-823e-ff691401bf61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f888793a-9c7d-40ee-823e-ff691401bf61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:31.364648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:31.365490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:31.366263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 19/34] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:28:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:31.372461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74ef3f-f376-4b00-9313-9cbc0bbd2404 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:31.476046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:31.511655 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74ef3f-f376-4b00-9313-9cbc0bbd2404 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:31.552491 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb220e4-3499-4e55-bbae-b74e86d74aa9 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:31.552884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb220e4-3499-4e55-bbae-b74e86d74aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:31.553153 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb220e4-3499-4e55-bbae-b74e86d74aa9 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:31.557547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 20/35] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:31.561444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a77ad-d28a-468b-b720-4098f6c45794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:31.561444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a77ad-d28a-468b-b720-4098f6c45794 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:31.561444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a77ad-d28a-468b-b720-4098f6c45794 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:31.561444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a77ad-d28a-468b-b720-4098f6c45794 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:31.561782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a77ad-d28a-468b-b720-4098f6c45794 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:31.561782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a77ad-d28a-468b-b720-4098f6c45794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:31.562475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a77ad-d28a-468b-b720-4098f6c45794 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:31.563568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1970-3f46-48bf-bbf9-718b0f49fcbf None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:31.564135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1970-3f46-48bf-bbf9-718b0f49fcbf None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:31.564598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c1970-3f46-48bf-bbf9-718b0f49fcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:31.573706 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 16/36] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:31.579335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a77ad-d28a-468b-b720-4098f6c45794 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:31.590548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-767c1970-3f46-48bf-bbf9-718b0f49fcbf None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:31.592024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-767c1970-3f46-48bf-bbf9-718b0f49fcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:31.674445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 21/37] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:31.678386 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c1970-3f46-48bf-bbf9-718b0f49fcbf None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:31.704006 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a77ad-d28a-468b-b720-4098f6c45794 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:31.746034 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6256f76-b928-41ff-b800-caea10a7c826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:31.747503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6256f76-b928-41ff-b800-caea10a7c826 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:31.747989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6256f76-b928-41ff-b800-caea10a7c826 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:31.748481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6256f76-b928-41ff-b800-caea10a7c826 None admin] PATH_INFO: `/v3/users/75330c8a27ec4acf96914b6033d2cd61` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:31.750567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6256f76-b928-41ff-b800-caea10a7c826 None admin] RBAC: Authorizing `identity:get_user(user_id=75330c8a27ec4acf96914b6033d2cd61)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:31.752340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6256f76-b928-41ff-b800-caea10a7c826 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:31.754345 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 17/38] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:31 2022] GET /identity/v3/users/75330c8a27ec4acf96914b6033d2cd61 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:31.762141 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6256f76-b928-41ff-b800-caea10a7c826 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:31.767118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:31.879488 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6256f76-b928-41ff-b800-caea10a7c826 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:31.899261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:31.919192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f53cc8-b378-486a-b8dd-8d00fef199d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:31.920033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f53cc8-b378-486a-b8dd-8d00fef199d5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:31.920461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f53cc8-b378-486a-b8dd-8d00fef199d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:31.920602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c1970-3f46-48bf-bbf9-718b0f49fcbf None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:31.921009 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f53cc8-b378-486a-b8dd-8d00fef199d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:31.922456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f53cc8-b378-486a-b8dd-8d00fef199d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:31.924160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f53cc8-b378-486a-b8dd-8d00fef199d5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:31.925885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 18/39] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:31.927157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b062fda-c2a8-444a-9f9f-d16858c34f7b None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:31.927720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b062fda-c2a8-444a-9f9f-d16858c34f7b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:31.927996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b062fda-c2a8-444a-9f9f-d16858c34f7b None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:31.930321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 19/40] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:31.932147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b062fda-c2a8-444a-9f9f-d16858c34f7b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:31.963319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4ac56-9721-4bd9-8436-6e592b860d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:31.963940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4ac56-9721-4bd9-8436-6e592b860d3d None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:31.964218 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4ac56-9721-4bd9-8436-6e592b860d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:31.964488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4ac56-9721-4bd9-8436-6e592b860d3d None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:31.965152 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4ac56-9721-4bd9-8436-6e592b860d3d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:31.966554 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4ac56-9721-4bd9-8436-6e592b860d3d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:31.981519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 22/41] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Oct 24 13:28:31 2022] POST /identity/v3/roles => generated 225 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:31.982746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cf69e-0832-4971-b7d4-c99208621be5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:31.983348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cf69e-0832-4971-b7d4-c99208621be5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:31.983348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cf69e-0832-4971-b7d4-c99208621be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:32.002433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fe8cf69e-0832-4971-b7d4-c99208621be5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:32.003558 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fe8cf69e-0832-4971-b7d4-c99208621be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:32.013682 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:32.051124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b062fda-c2a8-444a-9f9f-d16858c34f7b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:32.082591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 23/42] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:32.083443 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cf69e-0832-4971-b7d4-c99208621be5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:32.106694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8519da-764b-4d71-9c29-be4afca01380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:32.107328 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8519da-764b-4d71-9c29-be4afca01380 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:32.107606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8519da-764b-4d71-9c29-be4afca01380 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:32.107877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8519da-764b-4d71-9c29-be4afca01380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:32.108624 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8519da-764b-4d71-9c29-be4afca01380 None admin] RBAC: Adding query filter params (role.id=8b32d66a3cb3483c8ede361ad8637f66, scope.domain.id=default, user.id=75330c8a27ec4acf96914b6033d2cd61) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:32.108931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8519da-764b-4d71-9c29-be4afca01380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:32.110449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8519da-764b-4d71-9c29-be4afca01380 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:32.120749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 20/43] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Oct 24 13:28:31 2022] GET /identity/v3/role_assignments?user.id=75330c8a27ec4acf96914b6033d2cd61&scope.domain.id=default&role.id=8b32d66a3cb3483c8ede361ad8637f66 => generated 530 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:32.175548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:32.221594 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cf69e-0832-4971-b7d4-c99208621be5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:32.260828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3c86e-1f55-4dc2-989d-42fbfea99ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:32.261376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c86e-1f55-4dc2-989d-42fbfea99ab6 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:32.261659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c86e-1f55-4dc2-989d-42fbfea99ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:32.261935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c86e-1f55-4dc2-989d-42fbfea99ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:32.262822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3c86e-1f55-4dc2-989d-42fbfea99ab6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:32.264223 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3c86e-1f55-4dc2-989d-42fbfea99ab6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:32.281777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 24/44] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Oct 24 13:28:32 2022] POST /identity/v3/roles => generated 223 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:32.420078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8519da-764b-4d71-9c29-be4afca01380 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:32.521573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8519da-764b-4d71-9c29-be4afca01380 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:32.556256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a84343-bc59-44af-8fb5-f895aab987a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:32.556879 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84343-bc59-44af-8fb5-f895aab987a6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:32.557163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84343-bc59-44af-8fb5-f895aab987a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:32.557435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84343-bc59-44af-8fb5-f895aab987a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:32.558592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a84343-bc59-44af-8fb5-f895aab987a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:32.560034 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a84343-bc59-44af-8fb5-f895aab987a6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:32.561601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 21/45] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:32.567231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3c86e-1f55-4dc2-989d-42fbfea99ab6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:32.665818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3c86e-1f55-4dc2-989d-42fbfea99ab6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:32.692951 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:32.699519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9819c-d166-4906-b46a-2843d6cb36ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:32.700160 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9819c-d166-4906-b46a-2843d6cb36ce None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:32.700645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9819c-d166-4906-b46a-2843d6cb36ce None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:32.701094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9819c-d166-4906-b46a-2843d6cb36ce None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:32.701927 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9819c-d166-4906-b46a-2843d6cb36ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:32.703389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9819c-d166-4906-b46a-2843d6cb36ce None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:32.728520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 25/46] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:28:32 2022] POST /identity/v3/projects => generated 312 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:34.875833 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64081ff1-d841-47ba-ad5e-c6cde9c8b68a None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:34.876354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64081ff1-d841-47ba-ad5e-c6cde9c8b68a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:34.876354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64081ff1-d841-47ba-ad5e-c6cde9c8b68a None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:34.877725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 22/47] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:34.883564 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a2a76-0fb1-43f4-a5f8-2365bdb1ecc9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:34.883976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a2a76-0fb1-43f4-a5f8-2365bdb1ecc9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:34.884121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a2a76-0fb1-43f4-a5f8-2365bdb1ecc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:34.902854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e47a2a76-0fb1-43f4-a5f8-2365bdb1ecc9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:34.904028 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e47a2a76-0fb1-43f4-a5f8-2365bdb1ecc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:34.969030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 26/48] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:35.237265 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64081ff1-d841-47ba-ad5e-c6cde9c8b68a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:35.241143 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:35.336651 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64081ff1-d841-47ba-ad5e-c6cde9c8b68a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:35.368693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e4bb6-5f5f-4171-b056-3edc501630a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:35.369307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4bb6-5f5f-4171-b056-3edc501630a1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:35.369585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4bb6-5f5f-4171-b056-3edc501630a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:35.369851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4bb6-5f5f-4171-b056-3edc501630a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:35.370961 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e4bb6-5f5f-4171-b056-3edc501630a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:35.372389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e4bb6-5f5f-4171-b056-3edc501630a1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:35.373910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 23/49] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:35.379463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a2a76-0fb1-43f4-a5f8-2365bdb1ecc9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:35.477390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a2a76-0fb1-43f4-a5f8-2365bdb1ecc9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:35.489819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:35.512407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dcea6-ff5c-4b14-bdb1-76c04bb95d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:35.513016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dcea6-ff5c-4b14-bdb1-76c04bb95d78 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:35.513299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dcea6-ff5c-4b14-bdb1-76c04bb95d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:35.513575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dcea6-ff5c-4b14-bdb1-76c04bb95d78 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:35.514267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dcea6-ff5c-4b14-bdb1-76c04bb95d78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:35.515705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dcea6-ff5c-4b14-bdb1-76c04bb95d78 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:35.540194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 27/50] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Oct 24 13:28:35 2022] POST /identity/v3/projects => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:37.680129 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7de0e0-f0ba-43d1-8f18-8fb2d05402bf None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:37.680649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7de0e0-f0ba-43d1-8f18-8fb2d05402bf None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:37.680649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7de0e0-f0ba-43d1-8f18-8fb2d05402bf None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:37.681998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 24/51] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:37.687312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded0f82-3c69-4b70-9a3a-61fff9bc03b0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:37.687674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded0f82-3c69-4b70-9a3a-61fff9bc03b0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:37.687874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded0f82-3c69-4b70-9a3a-61fff9bc03b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:37.706610 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7ded0f82-3c69-4b70-9a3a-61fff9bc03b0 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:37.707803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7ded0f82-3c69-4b70-9a3a-61fff9bc03b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:37.771839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 28/52] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:38.045352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7de0e0-f0ba-43d1-8f18-8fb2d05402bf None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:38.114141 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:38.143227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7de0e0-f0ba-43d1-8f18-8fb2d05402bf None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:38.175684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4f155-ef87-46ab-a328-b6a8b67164a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:38.176302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4f155-ef87-46ab-a328-b6a8b67164a5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:38.176582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4f155-ef87-46ab-a328-b6a8b67164a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:38.176851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4f155-ef87-46ab-a328-b6a8b67164a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:38.177945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4f155-ef87-46ab-a328-b6a8b67164a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:38.180032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4f155-ef87-46ab-a328-b6a8b67164a5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:38.181568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 25/53] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:38.187001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded0f82-3c69-4b70-9a3a-61fff9bc03b0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:38.276745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:38.283483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded0f82-3c69-4b70-9a3a-61fff9bc03b0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:38.316433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4d8a4-708e-471e-9caa-5970ed772e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:38.317027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4d8a4-708e-471e-9caa-5970ed772e76 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:38.317304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4d8a4-708e-471e-9caa-5970ed772e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:38.317570 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4d8a4-708e-471e-9caa-5970ed772e76 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:38.318238 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4d8a4-708e-471e-9caa-5970ed772e76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:38.319692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4d8a4-708e-471e-9caa-5970ed772e76 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:38.345908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 29/54] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:28:38 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:28:41.547756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaeddca-c295-46d8-8721-7bf42b9042c9 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:41.547756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaeddca-c295-46d8-8721-7bf42b9042c9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:41.547756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaeddca-c295-46d8-8721-7bf42b9042c9 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:41.547756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 26/55] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:41.554453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acd9a3-a9c0-46cc-a6ac-bcb9c8c4a0c2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:41.555344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acd9a3-a9c0-46cc-a6ac-bcb9c8c4a0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:41.555839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acd9a3-a9c0-46cc-a6ac-bcb9c8c4a0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:41.575317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c3acd9a3-a9c0-46cc-a6ac-bcb9c8c4a0c2 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:41.576800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c3acd9a3-a9c0-46cc-a6ac-bcb9c8c4a0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:41.645883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 30/56] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:41.676175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ef344-e179-45ee-8ec7-a0dd4dd87628 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:41.676803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ef344-e179-45ee-8ec7-a0dd4dd87628 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:41.677276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ef344-e179-45ee-8ec7-a0dd4dd87628 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:41.679430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 27/57] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:41.685024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ffe6-e4f4-4655-9199-075c5dfdb2f2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:41.685317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ffe6-e4f4-4655-9199-075c5dfdb2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:41.685594 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ffe6-e4f4-4655-9199-075c5dfdb2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:41.712337 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-6134ffe6-e4f4-4655-9199-075c5dfdb2f2 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:41.713523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6134ffe6-e4f4-4655-9199-075c5dfdb2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:41.816713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 31/58] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:28:41.903269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20698fe9-5303-401b-8156-381ece207862 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:41.903913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20698fe9-5303-401b-8156-381ece207862 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:41.904383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20698fe9-5303-401b-8156-381ece207862 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:41.905904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 28/59] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:41.911366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7345de9-1698-4dd0-85f3-5199037b032b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:41.911662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7345de9-1698-4dd0-85f3-5199037b032b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:41.911943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7345de9-1698-4dd0-85f3-5199037b032b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:41.939189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c7345de9-1698-4dd0-85f3-5199037b032b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:41.940408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c7345de9-1698-4dd0-85f3-5199037b032b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:41.941125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20698fe9-5303-401b-8156-381ece207862 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:41.946113 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:42.059379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20698fe9-5303-401b-8156-381ece207862 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:42.076553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 32/60] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:42.108142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36758e-d695-48f1-ab09-40eb30d112dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:42.109115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36758e-d695-48f1-ab09-40eb30d112dc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.109589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36758e-d695-48f1-ab09-40eb30d112dc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.110065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36758e-d695-48f1-ab09-40eb30d112dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.136047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36758e-d695-48f1-ab09-40eb30d112dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6c36758e-d695-48f1-ab09-40eb30d112dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:42.150195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:42.151085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:42.151830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 29/61] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:28:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:28:42.152930 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:42.157687 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7345de9-1698-4dd0-85f3-5199037b032b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:42.187388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36758e-d695-48f1-ab09-40eb30d112dc None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:42.333089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7345de9-1698-4dd0-85f3-5199037b032b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:42.362683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36758e-d695-48f1-ab09-40eb30d112dc None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:42.379133 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f63d03-50f8-4533-9cd0-fa63178edf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:42.380177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f63d03-50f8-4533-9cd0-fa63178edf60 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.380932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f63d03-50f8-4533-9cd0-fa63178edf60 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.381211 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f63d03-50f8-4533-9cd0-fa63178edf60 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.381954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f63d03-50f8-4533-9cd0-fa63178edf60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:42.382283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f63d03-50f8-4533-9cd0-fa63178edf60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:42.383732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f63d03-50f8-4533-9cd0-fa63178edf60 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:42.396890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 33/62] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:42.398028 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf6343-c394-44fa-a347-39ab00bd3c47 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.398596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf6343-c394-44fa-a347-39ab00bd3c47 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.398876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf6343-c394-44fa-a347-39ab00bd3c47 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.400267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 34/63] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:42.402939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf6343-c394-44fa-a347-39ab00bd3c47 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:42.407620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:42.410811 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276bb77-9d15-426d-8a41-7ad8e412ffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:42.411454 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276bb77-9d15-426d-8a41-7ad8e412ffb5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.411736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276bb77-9d15-426d-8a41-7ad8e412ffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.412012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276bb77-9d15-426d-8a41-7ad8e412ffb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.423599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276bb77-9d15-426d-8a41-7ad8e412ffb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-4276bb77-9d15-426d-8a41-7ad8e412ffb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:42.428418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:42.430084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 30/64] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:28:42.430958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276bb77-9d15-426d-8a41-7ad8e412ffb5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:42.493108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:42.527311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf6343-c394-44fa-a347-39ab00bd3c47 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:42.535091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276bb77-9d15-426d-8a41-7ad8e412ffb5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:42.562843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940434f-3d04-4dbb-8fea-1ee73505a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:42.563478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940434f-3d04-4dbb-8fea-1ee73505a48a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.563753 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940434f-3d04-4dbb-8fea-1ee73505a48a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.564021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940434f-3d04-4dbb-8fea-1ee73505a48a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.568804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c0739-eafe-46bd-bd69-01289860a7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:42.569579 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c0739-eafe-46bd-bd69-01289860a7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.569965 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c0739-eafe-46bd-bd69-01289860a7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.570257 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c0739-eafe-46bd-bd69-01289860a7a2 None admin] PATH_INFO: `/v3/users/75330c8a27ec4acf96914b6033d2cd61` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.571384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c0739-eafe-46bd-bd69-01289860a7a2 None admin] RBAC: Authorizing `identity:get_user(user_id=75330c8a27ec4acf96914b6033d2cd61)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:42.572857 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c0739-eafe-46bd-bd69-01289860a7a2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:42.573892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940434f-3d04-4dbb-8fea-1ee73505a48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:42.574438 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 31/65] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:42 2022] GET /identity/v3/users/75330c8a27ec4acf96914b6033d2cd61 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:42.575601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33477-a36a-4572-b086-3620deb888a3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.575885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33477-a36a-4572-b086-3620deb888a3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.576161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33477-a36a-4572-b086-3620deb888a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-9940434f-3d04-4dbb-8fea-1ee73505a48a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:42.579217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:42.580155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 35/66] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:28:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:42.580906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940434f-3d04-4dbb-8fea-1ee73505a48a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:42.597482 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-dda33477-a36a-4572-b086-3620deb888a3 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:42.598637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-dda33477-a36a-4572-b086-3620deb888a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:42.641987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:42.684693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940434f-3d04-4dbb-8fea-1ee73505a48a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:42.720925 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5f8aa-c84c-416e-9b66-77769df93e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:42.721550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5f8aa-c84c-416e-9b66-77769df93e1e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.721831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5f8aa-c84c-416e-9b66-77769df93e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.722103 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5f8aa-c84c-416e-9b66-77769df93e1e None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.722807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5f8aa-c84c-416e-9b66-77769df93e1e None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:42.723112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5f8aa-c84c-416e-9b66-77769df93e1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:42.724529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5f8aa-c84c-416e-9b66-77769df93e1e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:42.734556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 36/67] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:28:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:42.735678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfec1dd-a7a5-4ef9-ace0-296d58d26fcb None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.735963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfec1dd-a7a5-4ef9-ace0-296d58d26fcb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.736235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfec1dd-a7a5-4ef9-ace0-296d58d26fcb None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.737599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 37/68] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:42.741393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfec1dd-a7a5-4ef9-ace0-296d58d26fcb None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:42.769122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 32/69] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:42.769970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda33477-a36a-4572-b086-3620deb888a3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:42.830031 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:42.863717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfec1dd-a7a5-4ef9-ace0-296d58d26fcb None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:42.881253 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda33477-a36a-4572-b086-3620deb888a3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:42.887622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:42.900197 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e3a65-6222-4312-84b5-e673a5ddf34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:42.900833 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e3a65-6222-4312-84b5-e673a5ddf34e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.901115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e3a65-6222-4312-84b5-e673a5ddf34e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.901387 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e3a65-6222-4312-84b5-e673a5ddf34e None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.902560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e3a65-6222-4312-84b5-e673a5ddf34e None admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:42.904009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e3a65-6222-4312-84b5-e673a5ddf34e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:42.905644 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 38/70] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:42 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:42.910401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e3a65-6222-4312-84b5-e673a5ddf34e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:42.923162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d9fea-6b03-4c6c-be74-6bd75024f680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:42.923776 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d9fea-6b03-4c6c-be74-6bd75024f680 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.924055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d9fea-6b03-4c6c-be74-6bd75024f680 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.924383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d9fea-6b03-4c6c-be74-6bd75024f680 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.924976 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d9fea-6b03-4c6c-be74-6bd75024f680 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:42.925488 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d9fea-6b03-4c6c-be74-6bd75024f680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:42.926678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d9fea-6b03-4c6c-be74-6bd75024f680 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:42.939407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 33/71] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:28:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:42.939407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661c257-5f5b-4ecc-be3f-cd722133b687 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:42.939407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661c257-5f5b-4ecc-be3f-cd722133b687 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:42.939407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661c257-5f5b-4ecc-be3f-cd722133b687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:42.954747 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a661c257-5f5b-4ecc-be3f-cd722133b687 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:42.956020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a661c257-5f5b-4ecc-be3f-cd722133b687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:43.018697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e3a65-6222-4312-84b5-e673a5ddf34e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:43.038681 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 34/72] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:43.038681 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661c257-5f5b-4ecc-be3f-cd722133b687 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:43.065454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb4dcc-71f8-4d08-aec2-7c35f2a228a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:43.066221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb4dcc-71f8-4d08-aec2-7c35f2a228a9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:43.066504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb4dcc-71f8-4d08-aec2-7c35f2a228a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:43.066828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb4dcc-71f8-4d08-aec2-7c35f2a228a9 None admin] PATH_INFO: `/v3/users/bca379d1a4aa4b3884c2c194379118dd` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:43.083469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb4dcc-71f8-4d08-aec2-7c35f2a228a9 None admin] RBAC: Authorizing `identity:get_user(user_id=bca379d1a4aa4b3884c2c194379118dd)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:43.085036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb4dcc-71f8-4d08-aec2-7c35f2a228a9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:43.088268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 39/73] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:42 2022] GET /identity/v3/users/bca379d1a4aa4b3884c2c194379118dd => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:43.088659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb4dcc-71f8-4d08-aec2-7c35f2a228a9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:43.091786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:43.160345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:43.164766 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661c257-5f5b-4ecc-be3f-cd722133b687 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:43.215702 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a264fbf-28b3-466c-a452-b159926bc4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:43.216396 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264fbf-28b3-466c-a452-b159926bc4c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:43.216679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264fbf-28b3-466c-a452-b159926bc4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:43.216951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264fbf-28b3-466c-a452-b159926bc4c9 None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790/users/75330c8a27ec4acf96914b6033d2cd61/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:43.220199 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a264fbf-28b3-466c-a452-b159926bc4c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a634485bda4247f4996f560aa4394790, user_id=75330c8a27ec4acf96914b6033d2cd61, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:43.221673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a264fbf-28b3-466c-a452-b159926bc4c9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:43.238842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 35/74] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:28:43 2022] PUT /identity/v3/projects/a634485bda4247f4996f560aa4394790/users/75330c8a27ec4acf96914b6033d2cd61/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:43.239671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a264fbf-28b3-466c-a452-b159926bc4c9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:43.248371 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb4dcc-71f8-4d08-aec2-7c35f2a228a9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:43.316407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc7ce2-54bf-4d7a-9db5-e738b044de70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:43.317607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc7ce2-54bf-4d7a-9db5-e738b044de70 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:43.317895 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc7ce2-54bf-4d7a-9db5-e738b044de70 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:43.318191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc7ce2-54bf-4d7a-9db5-e738b044de70 None admin] PATH_INFO: `/v3/users/bca379d1a4aa4b3884c2c194379118dd` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:43.320454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc7ce2-54bf-4d7a-9db5-e738b044de70 None admin] RBAC: Authorizing `identity:get_user(user_id=bca379d1a4aa4b3884c2c194379118dd)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:43.322323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc7ce2-54bf-4d7a-9db5-e738b044de70 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:43.324016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 40/75] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:43 2022] GET /identity/v3/users/bca379d1a4aa4b3884c2c194379118dd => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:43.324805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc7ce2-54bf-4d7a-9db5-e738b044de70 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:43.403312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a264fbf-28b3-466c-a452-b159926bc4c9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:43.424374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:43.439214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc7ce2-54bf-4d7a-9db5-e738b044de70 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:43.459168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6c33a-39c7-4648-994c-cfeec74b53a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:43.459813 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6c33a-39c7-4648-994c-cfeec74b53a5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:43.460613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6c33a-39c7-4648-994c-cfeec74b53a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:43.461037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6c33a-39c7-4648-994c-cfeec74b53a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:43.462232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6c33a-39c7-4648-994c-cfeec74b53a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:43.463937 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6c33a-39c7-4648-994c-cfeec74b53a5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:43.465168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:43.466897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 36/76] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:43.467497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6c33a-39c7-4648-994c-cfeec74b53a5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:43.489675 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c29d8-4c72-4095-b855-6abcf8807a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:43.491006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c29d8-4c72-4095-b855-6abcf8807a3e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:43.492478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c29d8-4c72-4095-b855-6abcf8807a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:43.493139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c29d8-4c72-4095-b855-6abcf8807a3e None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:43.494847 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c29d8-4c72-4095-b855-6abcf8807a3e None admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:43.496892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c29d8-4c72-4095-b855-6abcf8807a3e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:43.499307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 41/77] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:43 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:43.500419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c29d8-4c72-4095-b855-6abcf8807a3e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:43.598505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6c33a-39c7-4648-994c-cfeec74b53a5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:43.606840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c29d8-4c72-4095-b855-6abcf8807a3e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:43.641581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2da229-9f05-47f7-ade7-1ce7e130f9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:43.642345 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2da229-9f05-47f7-ade7-1ce7e130f9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:43.642517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94eeb7d-2d95-46f6-9106-79e80ecd757e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:43.642623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2da229-9f05-47f7-ade7-1ce7e130f9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:43.642884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2da229-9f05-47f7-ade7-1ce7e130f9d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:43.643142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eeb7d-2d95-46f6-9106-79e80ecd757e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:43.643427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eeb7d-2d95-46f6-9106-79e80ecd757e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:43.643704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94eeb7d-2d95-46f6-9106-79e80ecd757e None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:43.644834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94eeb7d-2d95-46f6-9106-79e80ecd757e None admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:43.646299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94eeb7d-2d95-46f6-9106-79e80ecd757e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:43.647948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 42/78] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:43 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:43.648789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94eeb7d-2d95-46f6-9106-79e80ecd757e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:43.654968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2da229-9f05-47f7-ade7-1ce7e130f9d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-1e2da229-9f05-47f7-ade7-1ce7e130f9d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:43.661426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:43.662312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:28:43.663222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:43.663808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 37/79] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:43.664335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2da229-9f05-47f7-ade7-1ce7e130f9d9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:43.708716 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:43.717651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:43.763336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94eeb7d-2d95-46f6-9106-79e80ecd757e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:43.778114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2da229-9f05-47f7-ade7-1ce7e130f9d9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:43.797822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2fd62-6d01-45be-b729-43f82973a0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:43.798698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2fd62-6d01-45be-b729-43f82973a0a5 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:43.798984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2fd62-6d01-45be-b729-43f82973a0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:43.799261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2fd62-6d01-45be-b729-43f82973a0a5 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:43.799916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2fd62-6d01-45be-b729-43f82973a0a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:43.801359 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2fd62-6d01-45be-b729-43f82973a0a5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:43.815596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7a73a-f72e-4cfa-b856-cd70447584e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:43.816457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7a73a-f72e-4cfa-b856-cd70447584e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:43.816833 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7a73a-f72e-4cfa-b856-cd70447584e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:43.817207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7a73a-f72e-4cfa-b856-cd70447584e3 None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790/users/bca379d1a4aa4b3884c2c194379118dd/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:43.820177 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7a73a-f72e-4cfa-b856-cd70447584e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a634485bda4247f4996f560aa4394790, user_id=bca379d1a4aa4b3884c2c194379118dd, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:43.821752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7a73a-f72e-4cfa-b856-cd70447584e3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:43.827008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 43/80] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:28:43 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:43.835536 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2fd62-6d01-45be-b729-43f82973a0a5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:43.837698 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 38/81] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:28:43 2022] PUT /identity/v3/projects/a634485bda4247f4996f560aa4394790/users/bca379d1a4aa4b3884c2c194379118dd/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:43.838960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7a73a-f72e-4cfa-b856-cd70447584e3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:43.967868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:43.974084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7a73a-f72e-4cfa-b856-cd70447584e3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:44.009291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2fd62-6d01-45be-b729-43f82973a0a5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:44.031238 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:44.072458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a6242-4a07-44bd-b456-944bafa5f51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:44.075186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a6242-4a07-44bd-b456-944bafa5f51b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:44.075473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a6242-4a07-44bd-b456-944bafa5f51b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:44.075754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a6242-4a07-44bd-b456-944bafa5f51b None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:44.076478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a6242-4a07-44bd-b456-944bafa5f51b None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:44.076757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a6242-4a07-44bd-b456-944bafa5f51b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:44.078269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a6242-4a07-44bd-b456-944bafa5f51b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:44.091495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf3b66-2e92-47c8-9406-f6a4067ea280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:44.092567 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf3b66-2e92-47c8-9406-f6a4067ea280 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:44.093048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf3b66-2e92-47c8-9406-f6a4067ea280 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:44.094425 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf3b66-2e92-47c8-9406-f6a4067ea280 None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790/users/bca379d1a4aa4b3884c2c194379118dd/roles/f948e6f6ee4f42de92195aee35b05641` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:44.099057 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf3b66-2e92-47c8-9406-f6a4067ea280 None admin] RBAC: Authorizing `identity:create_grant(project_id=a634485bda4247f4996f560aa4394790, user_id=bca379d1a4aa4b3884c2c194379118dd, role_id=f948e6f6ee4f42de92195aee35b05641)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:44.099877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 39/82] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:28:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:44.101598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf3b66-2e92-47c8-9406-f6a4067ea280 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:44.109017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a6242-4a07-44bd-b456-944bafa5f51b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:44.122316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 44/83] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:28:43 2022] PUT /identity/v3/projects/a634485bda4247f4996f560aa4394790/users/bca379d1a4aa4b3884c2c194379118dd/roles/f948e6f6ee4f42de92195aee35b05641 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:44.242902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a6242-4a07-44bd-b456-944bafa5f51b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:44.281248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a196da4-c493-4de3-8eaa-001dcfab9c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:44.281905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a196da4-c493-4de3-8eaa-001dcfab9c22 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:44.282212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a196da4-c493-4de3-8eaa-001dcfab9c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:44.282479 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a196da4-c493-4de3-8eaa-001dcfab9c22 None admin] PATH_INFO: `/v3/users/bca379d1a4aa4b3884c2c194379118dd` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:44.283696 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a196da4-c493-4de3-8eaa-001dcfab9c22 None admin] RBAC: Authorizing `identity:get_user(user_id=bca379d1a4aa4b3884c2c194379118dd)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:44.285159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a196da4-c493-4de3-8eaa-001dcfab9c22 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:44.286835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 40/84] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:44 2022] GET /identity/v3/users/bca379d1a4aa4b3884c2c194379118dd => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:44.293729 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf3b66-2e92-47c8-9406-f6a4067ea280 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:44.423303 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf3b66-2e92-47c8-9406-f6a4067ea280 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:44.468573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be5031-9cd4-4d1e-8fe1-b3be9a88d910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:44.469235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5031-9cd4-4d1e-8fe1-b3be9a88d910 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:44.469521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5031-9cd4-4d1e-8fe1-b3be9a88d910 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:44.469802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be5031-9cd4-4d1e-8fe1-b3be9a88d910 None admin] PATH_INFO: `/v3/projects/2b771dd639014f43b2724ae0d639ea4d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:44.471021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be5031-9cd4-4d1e-8fe1-b3be9a88d910 None admin] RBAC: Authorizing `identity:get_project(project_id=2b771dd639014f43b2724ae0d639ea4d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:44.472549 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be5031-9cd4-4d1e-8fe1-b3be9a88d910 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:44.474557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 45/85] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:44 2022] GET /identity/v3/projects/2b771dd639014f43b2724ae0d639ea4d => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:44.480593 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a196da4-c493-4de3-8eaa-001dcfab9c22 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:44.524485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:44.616230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a196da4-c493-4de3-8eaa-001dcfab9c22 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:44.658395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45970e58-6d36-47c8-9039-433f06d66344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:44.659065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970e58-6d36-47c8-9039-433f06d66344 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:44.659346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970e58-6d36-47c8-9039-433f06d66344 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:44.659620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970e58-6d36-47c8-9039-433f06d66344 None admin] PATH_INFO: `/v3/projects/2b771dd639014f43b2724ae0d639ea4d/users/bca379d1a4aa4b3884c2c194379118dd/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:44.661559 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45970e58-6d36-47c8-9039-433f06d66344 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b771dd639014f43b2724ae0d639ea4d, user_id=bca379d1a4aa4b3884c2c194379118dd, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:44.663554 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45970e58-6d36-47c8-9039-433f06d66344 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:44.686999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 41/86] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:28:44 2022] PUT /identity/v3/projects/2b771dd639014f43b2724ae0d639ea4d/users/bca379d1a4aa4b3884c2c194379118dd/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:45.656615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77771c-a7e5-4d5f-ad58-16f438323a83 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:45.657628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77771c-a7e5-4d5f-ad58-16f438323a83 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:45.658114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77771c-a7e5-4d5f-ad58-16f438323a83 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:45.659962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 46/87] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:45.665305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f86c35-74a7-47a3-a05a-c0e4419805eb None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:45.665758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f86c35-74a7-47a3-a05a-c0e4419805eb None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:45.665937 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f86c35-74a7-47a3-a05a-c0e4419805eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:45.692208 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-44f86c35-74a7-47a3-a05a-c0e4419805eb None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:45.693457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-44f86c35-74a7-47a3-a05a-c0e4419805eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:45.776705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 42/88] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:46.318201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77771c-a7e5-4d5f-ad58-16f438323a83 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:46.365018 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:46.443408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77771c-a7e5-4d5f-ad58-16f438323a83 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:46.489616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391399bd-008f-4b41-bd52-c8b6641b897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:46.490201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391399bd-008f-4b41-bd52-c8b6641b897d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:46.490492 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391399bd-008f-4b41-bd52-c8b6641b897d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:46.490826 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391399bd-008f-4b41-bd52-c8b6641b897d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:46.508124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391399bd-008f-4b41-bd52-c8b6641b897d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-391399bd-008f-4b41-bd52-c8b6641b897d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:46.513530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:46.514502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:46.515270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:46.519481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 47/89] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:28:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:46.520826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f86c35-74a7-47a3-a05a-c0e4419805eb None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:46.546647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:46.640040 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f86c35-74a7-47a3-a05a-c0e4419805eb None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:46.685702 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105853b-54a1-45d8-b82c-290c2a1ca340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:46.686402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105853b-54a1-45d8-b82c-290c2a1ca340 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:46.686634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105853b-54a1-45d8-b82c-290c2a1ca340 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:46.686882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105853b-54a1-45d8-b82c-290c2a1ca340 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:46.687551 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105853b-54a1-45d8-b82c-290c2a1ca340 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:46.687855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105853b-54a1-45d8-b82c-290c2a1ca340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:46.689238 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105853b-54a1-45d8-b82c-290c2a1ca340 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:46.698736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 43/90] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:46.704478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391399bd-008f-4b41-bd52-c8b6641b897d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:46.846675 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:46.851173 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391399bd-008f-4b41-bd52-c8b6641b897d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:46.898888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b4714-2bf4-4e70-9d0b-6d77784066fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:46.899547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b4714-2bf4-4e70-9d0b-6d77784066fe None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:46.899830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b4714-2bf4-4e70-9d0b-6d77784066fe None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:46.900109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b4714-2bf4-4e70-9d0b-6d77784066fe None admin] PATH_INFO: `/v3/users/75330c8a27ec4acf96914b6033d2cd61` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:46.902997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b4714-2bf4-4e70-9d0b-6d77784066fe None admin] RBAC: Authorizing `identity:get_user(user_id=75330c8a27ec4acf96914b6033d2cd61)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:46.904453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b4714-2bf4-4e70-9d0b-6d77784066fe None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:46.906056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 48/91] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:46 2022] GET /identity/v3/users/75330c8a27ec4acf96914b6033d2cd61 => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:46.911555 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105853b-54a1-45d8-b82c-290c2a1ca340 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:47.019640 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:47.040498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105853b-54a1-45d8-b82c-290c2a1ca340 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:47.070140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7973a-c611-4e81-b003-5abf71d9bab5 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.070484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7973a-c611-4e81-b003-5abf71d9bab5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.070723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7973a-c611-4e81-b003-5abf71d9bab5 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.072628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 49/92] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:47.078819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1c4a-2c70-42aa-93f5-fd1f98221448 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.079110 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1c4a-2c70-42aa-93f5-fd1f98221448 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.079385 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1c4a-2c70-42aa-93f5-fd1f98221448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.081651 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9186a8c-3c0d-4b97-bc8c-c9e588edf796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:47.082300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9186a8c-3c0d-4b97-bc8c-c9e588edf796 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.082588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9186a8c-3c0d-4b97-bc8c-c9e588edf796 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.082860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9186a8c-3c0d-4b97-bc8c-c9e588edf796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.101472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0a9a1c4a-2c70-42aa-93f5-fd1f98221448 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:47.103097 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0a9a1c4a-2c70-42aa-93f5-fd1f98221448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:47.176413 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9186a8c-3c0d-4b97-bc8c-c9e588edf796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:47.177884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9186a8c-3c0d-4b97-bc8c-c9e588edf796 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:47.192911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 50/93] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:47.194845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b45f2-7d95-4555-849c-0af5803968de None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.195347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b45f2-7d95-4555-849c-0af5803968de None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.195822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b45f2-7d95-4555-849c-0af5803968de None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.197503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 51/94] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:47.214505 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b50bd-3e9b-44ba-911f-7038ffcca9ef None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.214921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b50bd-3e9b-44ba-911f-7038ffcca9ef None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.215207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b50bd-3e9b-44ba-911f-7038ffcca9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.263236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f99b50bd-3e9b-44ba-911f-7038ffcca9ef None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:47.264619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f99b50bd-3e9b-44ba-911f-7038ffcca9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:47.275048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:47.294112 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 44/95] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:28:47.300630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9186a8c-3c0d-4b97-bc8c-c9e588edf796 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:47.392579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 52/96] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:47.461577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9186a8c-3c0d-4b97-bc8c-c9e588edf796 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:47.505500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3b3bd-5ef0-454f-b571-c9e9ff26dbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:47.506136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3b3bd-5ef0-454f-b571-c9e9ff26dbd6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.506447 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3b3bd-5ef0-454f-b571-c9e9ff26dbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.506760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3b3bd-5ef0-454f-b571-c9e9ff26dbd6 None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.508113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3b3bd-5ef0-454f-b571-c9e9ff26dbd6 None admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:47.509542 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3b3bd-5ef0-454f-b571-c9e9ff26dbd6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:47.511194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 45/97] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:47 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:47.517101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b50bd-3e9b-44ba-911f-7038ffcca9ef None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:47.560957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:47.647534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b50bd-3e9b-44ba-911f-7038ffcca9ef None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:47.650709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3b3bd-5ef0-454f-b571-c9e9ff26dbd6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:47.689155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bfa38-8d96-47dc-a5b8-2583ef65fb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:47.689976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bfa38-8d96-47dc-a5b8-2583ef65fb9b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.690297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bfa38-8d96-47dc-a5b8-2583ef65fb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.690581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bfa38-8d96-47dc-a5b8-2583ef65fb9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.691708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bfa38-8d96-47dc-a5b8-2583ef65fb9b None admin] RBAC: Adding query filter params (role.id=8b32d66a3cb3483c8ede361ad8637f66, scope.project.id=a634485bda4247f4996f560aa4394790, user.id=75330c8a27ec4acf96914b6033d2cd61) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:47.692031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bfa38-8d96-47dc-a5b8-2583ef65fb9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:47.693564 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bfa38-8d96-47dc-a5b8-2583ef65fb9b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:47.701887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 53/98] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:28:47 2022] GET /identity/v3/role_assignments?user.id=75330c8a27ec4acf96914b6033d2cd61&scope.project.id=a634485bda4247f4996f560aa4394790&role.id=8b32d66a3cb3483c8ede361ad8637f66 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:47.763748 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44558b85-2b95-482f-a9e7-bbe822dcf029 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.764598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44558b85-2b95-482f-a9e7-bbe822dcf029 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.764889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44558b85-2b95-482f-a9e7-bbe822dcf029 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.768582 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 54/99] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:47.773380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:47.775005 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf038-a1c9-420e-99ea-ce9e8142d39c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.775300 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf038-a1c9-420e-99ea-ce9e8142d39c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.775583 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf038-a1c9-420e-99ea-ce9e8142d39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.803365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-299bf038-a1c9-420e-99ea-ce9e8142d39c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:47.805190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-299bf038-a1c9-420e-99ea-ce9e8142d39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:47.853921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3b3bd-5ef0-454f-b571-c9e9ff26dbd6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:47.895398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 55/100] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:28:47.897013 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b09ba-319e-4489-89ee-de405c5e3a0a None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.897518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b09ba-319e-4489-89ee-de405c5e3a0a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.898051 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b09ba-319e-4489-89ee-de405c5e3a0a None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.899029 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c79cbe-f334-4e45-97bf-448e1e1663da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:47.899774 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c79cbe-f334-4e45-97bf-448e1e1663da None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.900046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 56/101] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:47.900145 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c79cbe-f334-4e45-97bf-448e1e1663da None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.901094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b09ba-319e-4489-89ee-de405c5e3a0a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:47.902393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c79cbe-f334-4e45-97bf-448e1e1663da None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.917289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c79cbe-f334-4e45-97bf-448e1e1663da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-a3c79cbe-f334-4e45-97bf-448e1e1663da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:47.922795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:47.923644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:28:47.924478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:47.926862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 46/102] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:47.928346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c691a-698c-46cb-bfc6-620e7429e1ae None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:47.929727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c691a-698c-46cb-bfc6-620e7429e1ae None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:47.930893 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c691a-698c-46cb-bfc6-620e7429e1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:47.936311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:47.950501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e41c691a-698c-46cb-bfc6-620e7429e1ae None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:47.952008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e41c691a-698c-46cb-bfc6-620e7429e1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:48.049102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 47/103] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:48.049981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c691a-698c-46cb-bfc6-620e7429e1ae None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:48.055910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b09ba-319e-4489-89ee-de405c5e3a0a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:48.122569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:48.134457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e12ad6-9bf6-46ea-ad7b-fd33128c90f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:48.135352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e12ad6-9bf6-46ea-ad7b-fd33128c90f4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:48.135828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e12ad6-9bf6-46ea-ad7b-fd33128c90f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:48.136272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e12ad6-9bf6-46ea-ad7b-fd33128c90f4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:48.160136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e12ad6-9bf6-46ea-ad7b-fd33128c90f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b8e12ad6-9bf6-46ea-ad7b-fd33128c90f4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:48.172119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:48.172975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:28:48.173705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:48.177897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 57/104] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:48.184170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e12ad6-9bf6-46ea-ad7b-fd33128c90f4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:48.235815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c691a-698c-46cb-bfc6-620e7429e1ae None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:48.277934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b3b0e-057a-4ac9-8cd3-3448499ab9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:48.278729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b3b0e-057a-4ac9-8cd3-3448499ab9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:48.279010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b3b0e-057a-4ac9-8cd3-3448499ab9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:48.279288 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b3b0e-057a-4ac9-8cd3-3448499ab9d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:48.280438 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b3b0e-057a-4ac9-8cd3-3448499ab9d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:48.280910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b3b0e-057a-4ac9-8cd3-3448499ab9d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:48.282601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b3b0e-057a-4ac9-8cd3-3448499ab9d9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:48.293033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 48/105] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:28:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:48.293678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b3b0e-057a-4ac9-8cd3-3448499ab9d9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:48.296622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:48.333839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e12ad6-9bf6-46ea-ad7b-fd33128c90f4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:48.384061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9401e-bcf4-4786-8d36-aaeece74187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:48.385499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9401e-bcf4-4786-8d36-aaeece74187e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:48.385886 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9401e-bcf4-4786-8d36-aaeece74187e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:48.386301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9401e-bcf4-4786-8d36-aaeece74187e None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:48.387561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9401e-bcf4-4786-8d36-aaeece74187e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:48.387980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9401e-bcf4-4786-8d36-aaeece74187e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:48.389854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9401e-bcf4-4786-8d36-aaeece74187e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:48.400280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 58/106] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:28:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:48.401381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9401e-bcf4-4786-8d36-aaeece74187e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:48.411653 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:48.415978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b3b0e-057a-4ac9-8cd3-3448499ab9d9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:48.461635 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4026ec-5b08-4a2c-8ffd-0703de390359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:48.462456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4026ec-5b08-4a2c-8ffd-0703de390359 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:48.462745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4026ec-5b08-4a2c-8ffd-0703de390359 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:48.463016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4026ec-5b08-4a2c-8ffd-0703de390359 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:48.473277 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4026ec-5b08-4a2c-8ffd-0703de390359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-ce4026ec-5b08-4a2c-8ffd-0703de390359 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:48.478805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:48.479656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:48.480424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 49/107] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:28:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:48.481037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4026ec-5b08-4a2c-8ffd-0703de390359 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:48.511004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9401e-bcf4-4786-8d36-aaeece74187e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:48.544030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cecc9-49ad-4fdc-8adf-198b6d60c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:48.544657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cecc9-49ad-4fdc-8adf-198b6d60c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:48.545026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cecc9-49ad-4fdc-8adf-198b6d60c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:48.545301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cecc9-49ad-4fdc-8adf-198b6d60c8d4 None admin] PATH_INFO: `/v3/users/bca379d1a4aa4b3884c2c194379118dd` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:48.546453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cecc9-49ad-4fdc-8adf-198b6d60c8d4 None admin] RBAC: Authorizing `identity:get_user(user_id=bca379d1a4aa4b3884c2c194379118dd)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:48.547889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cecc9-49ad-4fdc-8adf-198b6d60c8d4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:48.549483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 59/108] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:48 2022] GET /identity/v3/users/bca379d1a4aa4b3884c2c194379118dd => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:48.550263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cecc9-49ad-4fdc-8adf-198b6d60c8d4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:48.562277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:48.586384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4026ec-5b08-4a2c-8ffd-0703de390359 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:48.618778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ca1df-4897-46ca-8258-bd9dfa9a7dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:48.619403 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ca1df-4897-46ca-8258-bd9dfa9a7dcf None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:48.620341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ca1df-4897-46ca-8258-bd9dfa9a7dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:48.620341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ca1df-4897-46ca-8258-bd9dfa9a7dcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:48.621056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ca1df-4897-46ca-8258-bd9dfa9a7dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:48.622544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ca1df-4897-46ca-8258-bd9dfa9a7dcf None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:48.624099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 50/109] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:48.624847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ca1df-4897-46ca-8258-bd9dfa9a7dcf None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:48.651934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cecc9-49ad-4fdc-8adf-198b6d60c8d4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:48.653652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:48.685148 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc8d2b-58a6-476c-adc6-73b4a4ff8291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:48.685707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc8d2b-58a6-476c-adc6-73b4a4ff8291 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:48.685988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc8d2b-58a6-476c-adc6-73b4a4ff8291 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:48.686291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc8d2b-58a6-476c-adc6-73b4a4ff8291 None admin] PATH_INFO: `/v3/users/fb194d5098de4c589de2e4386923b8a4` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:48.700261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc8d2b-58a6-476c-adc6-73b4a4ff8291 None admin] RBAC: Authorizing `identity:get_user(user_id=fb194d5098de4c589de2e4386923b8a4)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:48.701718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc8d2b-58a6-476c-adc6-73b4a4ff8291 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:48.703358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 60/110] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:48 2022] GET /identity/v3/users/fb194d5098de4c589de2e4386923b8a4 => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:48.704493 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96435d-1d32-4a53-beb6-9fd865653501 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:48.704780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96435d-1d32-4a53-beb6-9fd865653501 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:48.705052 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96435d-1d32-4a53-beb6-9fd865653501 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:48.706273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 61/111] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:48.706972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96435d-1d32-4a53-beb6-9fd865653501 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:48.730382 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ca1df-4897-46ca-8258-bd9dfa9a7dcf None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:48.764355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a023d8-bf79-40ad-a0ce-1d5d906fb6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:48.764962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a023d8-bf79-40ad-a0ce-1d5d906fb6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:48.765249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a023d8-bf79-40ad-a0ce-1d5d906fb6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:48.765519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a023d8-bf79-40ad-a0ce-1d5d906fb6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:48.766211 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a023d8-bf79-40ad-a0ce-1d5d906fb6e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:48.766519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a023d8-bf79-40ad-a0ce-1d5d906fb6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:48.767895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a023d8-bf79-40ad-a0ce-1d5d906fb6e8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:48.775867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 51/112] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:28:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:48.776759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a023d8-bf79-40ad-a0ce-1d5d906fb6e8 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:48.781834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:48.816773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96435d-1d32-4a53-beb6-9fd865653501 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:48.851558 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627d0e8-6488-4f39-a9cf-38f5e0ff5d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:48.852169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627d0e8-6488-4f39-a9cf-38f5e0ff5d3f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:48.852453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627d0e8-6488-4f39-a9cf-38f5e0ff5d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:48.852729 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627d0e8-6488-4f39-a9cf-38f5e0ff5d3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:48.867833 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:48.880165 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a023d8-bf79-40ad-a0ce-1d5d906fb6e8 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:48.914112 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e48a7-1a24-4e9e-9731-7f7dacce336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:48.914754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e48a7-1a24-4e9e-9731-7f7dacce336d None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:48.915031 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e48a7-1a24-4e9e-9731-7f7dacce336d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:48.915296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e48a7-1a24-4e9e-9731-7f7dacce336d None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:48.915939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e48a7-1a24-4e9e-9731-7f7dacce336d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:48.917630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e48a7-1a24-4e9e-9731-7f7dacce336d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:48.920278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627d0e8-6488-4f39-a9cf-38f5e0ff5d3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:48.921721 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627d0e8-6488-4f39-a9cf-38f5e0ff5d3f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:48.949862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 52/113] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:28:48 2022] POST /identity/v3/projects => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:48.950477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e48a7-1a24-4e9e-9731-7f7dacce336d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:49.001441 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:49.040801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 62/114] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:28:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:28:49.042446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2cd9e0-d674-4586-b75e-d5d58da2f9c4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:49.042732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2cd9e0-d674-4586-b75e-d5d58da2f9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:49.043001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2cd9e0-d674-4586-b75e-d5d58da2f9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:49.063588 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ca2cd9e0-d674-4586-b75e-d5d58da2f9c4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:49.064902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ca2cd9e0-d674-4586-b75e-d5d58da2f9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:49.079135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e48a7-1a24-4e9e-9731-7f7dacce336d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:49.115550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baa873-2c4a-445f-b31b-7a96f3830859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:49.116753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baa873-2c4a-445f-b31b-7a96f3830859 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:49.117209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baa873-2c4a-445f-b31b-7a96f3830859 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:49.117657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baa873-2c4a-445f-b31b-7a96f3830859 None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:49.118764 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47baa873-2c4a-445f-b31b-7a96f3830859 None admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:49.120285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47baa873-2c4a-445f-b31b-7a96f3830859 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:49.121954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 53/115] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:48 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:49.123104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baa873-2c4a-445f-b31b-7a96f3830859 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:49.137197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:49.146886 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 63/116] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:49.148172 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2cd9e0-d674-4586-b75e-d5d58da2f9c4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:49.237582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baa873-2c4a-445f-b31b-7a96f3830859 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:49.274953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:49.277226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab350164-6a43-420f-ac2d-3707e9d4efde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:49.277867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab350164-6a43-420f-ac2d-3707e9d4efde None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:49.278144 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab350164-6a43-420f-ac2d-3707e9d4efde None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:49.278449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab350164-6a43-420f-ac2d-3707e9d4efde None admin] PATH_INFO: `/v3/users/bca379d1a4aa4b3884c2c194379118dd` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:49.279685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab350164-6a43-420f-ac2d-3707e9d4efde None admin] RBAC: Authorizing `identity:get_user(user_id=bca379d1a4aa4b3884c2c194379118dd)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:49.281137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab350164-6a43-420f-ac2d-3707e9d4efde None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:49.282770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 54/117] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:49 2022] GET /identity/v3/users/bca379d1a4aa4b3884c2c194379118dd => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:49.283500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab350164-6a43-420f-ac2d-3707e9d4efde None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:49.286679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2cd9e0-d674-4586-b75e-d5d58da2f9c4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:49.328710 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb8768-f11c-4e30-b621-964c98911fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:49.329451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb8768-f11c-4e30-b621-964c98911fff None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:49.329800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb8768-f11c-4e30-b621-964c98911fff None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:49.330142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb8768-f11c-4e30-b621-964c98911fff None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:49.331789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb8768-f11c-4e30-b621-964c98911fff None admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:49.333753 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb8768-f11c-4e30-b621-964c98911fff None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:49.337102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 64/118] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:49 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:49.337975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb8768-f11c-4e30-b621-964c98911fff None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:49.372460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:49.399451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab350164-6a43-420f-ac2d-3707e9d4efde None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:49.438362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152726f-0f17-47f8-bc62-23ef021a99a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:49.439402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152726f-0f17-47f8-bc62-23ef021a99a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:49.439686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152726f-0f17-47f8-bc62-23ef021a99a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:49.439955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152726f-0f17-47f8-bc62-23ef021a99a6 None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790/users/fb194d5098de4c589de2e4386923b8a4/roles/84fba578b2cd41e6958097aec20a6f6e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:49.441976 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152726f-0f17-47f8-bc62-23ef021a99a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a634485bda4247f4996f560aa4394790, user_id=fb194d5098de4c589de2e4386923b8a4, role_id=84fba578b2cd41e6958097aec20a6f6e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:49.443476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152726f-0f17-47f8-bc62-23ef021a99a6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:49.458093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 55/119] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:28:49 2022] PUT /identity/v3/projects/a634485bda4247f4996f560aa4394790/users/fb194d5098de4c589de2e4386923b8a4/roles/84fba578b2cd41e6958097aec20a6f6e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:49.459130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152726f-0f17-47f8-bc62-23ef021a99a6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:49.459350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb8768-f11c-4e30-b621-964c98911fff None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:49.504737 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47824ea0-b763-4d55-9995-1f0cef647018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:49.505721 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47824ea0-b763-4d55-9995-1f0cef647018 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:49.506022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47824ea0-b763-4d55-9995-1f0cef647018 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:49.506334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47824ea0-b763-4d55-9995-1f0cef647018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:49.555890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:49.597307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47824ea0-b763-4d55-9995-1f0cef647018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:49.598872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47824ea0-b763-4d55-9995-1f0cef647018 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:49.617030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152726f-0f17-47f8-bc62-23ef021a99a6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:49.667053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2d4e-74be-4cc6-83e8-4bb3b510b88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:49.667675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2d4e-74be-4cc6-83e8-4bb3b510b88e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:49.667958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2d4e-74be-4cc6-83e8-4bb3b510b88e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:49.668230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2d4e-74be-4cc6-83e8-4bb3b510b88e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:49.668960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a2d4e-74be-4cc6-83e8-4bb3b510b88e None admin] RBAC: Adding query filter params (role.id=09a932be0c1b481c8494705d2030fc2a, scope.project.id=a634485bda4247f4996f560aa4394790, user.id=bca379d1a4aa4b3884c2c194379118dd) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:49.669263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a2d4e-74be-4cc6-83e8-4bb3b510b88e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:49.671384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a2d4e-74be-4cc6-83e8-4bb3b510b88e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:49.677841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 56/120] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:28:49 2022] GET /identity/v3/role_assignments?user.id=bca379d1a4aa4b3884c2c194379118dd&scope.project.id=a634485bda4247f4996f560aa4394790&role.id=09a932be0c1b481c8494705d2030fc2a => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:49.679218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2d4e-74be-4cc6-83e8-4bb3b510b88e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:49.683209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:49.717223 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 65/121] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:28:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:28:49.722768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47824ea0-b763-4d55-9995-1f0cef647018 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:49.852461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:49.877327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2d4e-74be-4cc6-83e8-4bb3b510b88e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:49.920937 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47824ea0-b763-4d55-9995-1f0cef647018 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:49.937659 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e17a66-071a-4c90-a091-ad0e043c775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:49.938743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e17a66-071a-4c90-a091-ad0e043c775e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:49.939219 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e17a66-071a-4c90-a091-ad0e043c775e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:49.939692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e17a66-071a-4c90-a091-ad0e043c775e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:49.954773 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e17a66-071a-4c90-a091-ad0e043c775e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-b7e17a66-071a-4c90-a091-ad0e043c775e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:49.962213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:49.963119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:49.966877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 57/122] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:49.971112 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e17a66-071a-4c90-a091-ad0e043c775e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:49.973258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5151e-1654-4b79-9f16-d1f418486fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:49.973904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5151e-1654-4b79-9f16-d1f418486fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:49.974203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5151e-1654-4b79-9f16-d1f418486fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:49.974481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5151e-1654-4b79-9f16-d1f418486fb6 None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:49.975652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5151e-1654-4b79-9f16-d1f418486fb6 None admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:49.977453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5151e-1654-4b79-9f16-d1f418486fb6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:49.978980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 66/123] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:49 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:49.985430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5151e-1654-4b79-9f16-d1f418486fb6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:50.042216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:50.090719 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e17a66-071a-4c90-a091-ad0e043c775e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:50.106696 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5151e-1654-4b79-9f16-d1f418486fb6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:50.127921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa38ee-3f0a-4b9d-8477-7d90c6b84321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:50.128916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa38ee-3f0a-4b9d-8477-7d90c6b84321 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:50.129423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa38ee-3f0a-4b9d-8477-7d90c6b84321 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:50.129898 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa38ee-3f0a-4b9d-8477-7d90c6b84321 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:50.130794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa38ee-3f0a-4b9d-8477-7d90c6b84321 None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:50.131299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa38ee-3f0a-4b9d-8477-7d90c6b84321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:50.133629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa38ee-3f0a-4b9d-8477-7d90c6b84321 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:50.143195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 58/124] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:28:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:50.143586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fadd7-cf01-4141-887e-bf4aafb5d856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:50.145821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fadd7-cf01-4141-887e-bf4aafb5d856 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:50.145821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fadd7-cf01-4141-887e-bf4aafb5d856 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:50.145821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fadd7-cf01-4141-887e-bf4aafb5d856 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:50.148409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fadd7-cf01-4141-887e-bf4aafb5d856 None admin] RBAC: Adding query filter params (role.id=f948e6f6ee4f42de92195aee35b05641, scope.project.id=a634485bda4247f4996f560aa4394790, user.id=bca379d1a4aa4b3884c2c194379118dd) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:50.148409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fadd7-cf01-4141-887e-bf4aafb5d856 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:50.148659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fadd7-cf01-4141-887e-bf4aafb5d856 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:50.149685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa38ee-3f0a-4b9d-8477-7d90c6b84321 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:50.155135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 67/125] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:28:49 2022] GET /identity/v3/role_assignments?user.id=bca379d1a4aa4b3884c2c194379118dd&scope.project.id=a634485bda4247f4996f560aa4394790&role.id=f948e6f6ee4f42de92195aee35b05641 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:50.155629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:50.290309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa38ee-3f0a-4b9d-8477-7d90c6b84321 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:50.318765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:50.333868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c1904-60ae-4595-8222-fc654e3803e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:50.335394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1904-60ae-4595-8222-fc654e3803e3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:50.335880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1904-60ae-4595-8222-fc654e3803e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:50.336326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1904-60ae-4595-8222-fc654e3803e3 None admin] PATH_INFO: `/v3/users/bca379d1a4aa4b3884c2c194379118dd` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:50.337714 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c1904-60ae-4595-8222-fc654e3803e3 None admin] RBAC: Authorizing `identity:get_user(user_id=bca379d1a4aa4b3884c2c194379118dd)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:50.339378 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c1904-60ae-4595-8222-fc654e3803e3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:50.341603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 59/126] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:50 2022] GET /identity/v3/users/bca379d1a4aa4b3884c2c194379118dd => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:50.347229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fadd7-cf01-4141-887e-bf4aafb5d856 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:50.454919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fadd7-cf01-4141-887e-bf4aafb5d856 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:50.488977 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4286b-c517-4096-80d5-a4031c6754c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:50.489580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4286b-c517-4096-80d5-a4031c6754c2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:50.489857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4286b-c517-4096-80d5-a4031c6754c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:50.490125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4286b-c517-4096-80d5-a4031c6754c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:50.557136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:50.561700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4286b-c517-4096-80d5-a4031c6754c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:50.563185 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4286b-c517-4096-80d5-a4031c6754c2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:50.660471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 68/127] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:28:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 24 13:28:50.666597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c1904-60ae-4595-8222-fc654e3803e3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:50.767636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c1904-60ae-4595-8222-fc654e3803e3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:50.800677 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4f94e-b8f5-4624-b3e8-5b9a4c57492c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:50.801313 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4f94e-b8f5-4624-b3e8-5b9a4c57492c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:50.801592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4f94e-b8f5-4624-b3e8-5b9a4c57492c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:50.801865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4f94e-b8f5-4624-b3e8-5b9a4c57492c None admin] PATH_INFO: `/v3/projects/2b771dd639014f43b2724ae0d639ea4d` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:50.803039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4f94e-b8f5-4624-b3e8-5b9a4c57492c None admin] RBAC: Authorizing `identity:get_project(project_id=2b771dd639014f43b2724ae0d639ea4d)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:50.805045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4f94e-b8f5-4624-b3e8-5b9a4c57492c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:50.806863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 60/128] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:50 2022] GET /identity/v3/projects/2b771dd639014f43b2724ae0d639ea4d => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:50.812503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4286b-c517-4096-80d5-a4031c6754c2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:50.914079 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:50.919186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4286b-c517-4096-80d5-a4031c6754c2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:50.954656 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5050b1-9ccc-4191-9a52-ca1232fdc81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:50.955266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5050b1-9ccc-4191-9a52-ca1232fdc81e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:50.955547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5050b1-9ccc-4191-9a52-ca1232fdc81e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:50.955820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5050b1-9ccc-4191-9a52-ca1232fdc81e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:50.956600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5050b1-9ccc-4191-9a52-ca1232fdc81e None admin] RBAC: Adding query filter params (role.id=09a932be0c1b481c8494705d2030fc2a, scope.project.id=2b771dd639014f43b2724ae0d639ea4d, user.id=bca379d1a4aa4b3884c2c194379118dd) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:50.956907 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5050b1-9ccc-4191-9a52-ca1232fdc81e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:50.958424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5050b1-9ccc-4191-9a52-ca1232fdc81e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:50.964917 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 69/129] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:28:50 2022] GET /identity/v3/role_assignments?user.id=bca379d1a4aa4b3884c2c194379118dd&scope.project.id=2b771dd639014f43b2724ae0d639ea4d&role.id=09a932be0c1b481c8494705d2030fc2a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:51.272546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73a663-5247-4e96-8b41-7445a476f2bc None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:51.273132 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73a663-5247-4e96-8b41-7445a476f2bc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:51.273132 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73a663-5247-4e96-8b41-7445a476f2bc None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:51.274471 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 61/130] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:51.279754 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c97cf-587f-48e1-b45f-1fd1f7bdd937 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:51.280042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c97cf-587f-48e1-b45f-1fd1f7bdd937 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:51.280317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c97cf-587f-48e1-b45f-1fd1f7bdd937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:51.298556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-885c97cf-587f-48e1-b45f-1fd1f7bdd937 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:51.299734 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-885c97cf-587f-48e1-b45f-1fd1f7bdd937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:51.368164 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 70/131] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:51.640032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73a663-5247-4e96-8b41-7445a476f2bc None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:51.702872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:51.746006 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73a663-5247-4e96-8b41-7445a476f2bc None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:51.778705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c28c5-f85b-4693-b953-c9dae33ac2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:51.779330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c28c5-f85b-4693-b953-c9dae33ac2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:51.779609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c28c5-f85b-4693-b953-c9dae33ac2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:51.779878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c28c5-f85b-4693-b953-c9dae33ac2c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:51.780967 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c28c5-f85b-4693-b953-c9dae33ac2c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:51.782434 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c28c5-f85b-4693-b953-c9dae33ac2c6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:51.783999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 62/132] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:51.789899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c97cf-587f-48e1-b45f-1fd1f7bdd937 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:51.803229 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed17028-ab43-42d3-8431-0ebdc2c79d9b None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:51.804045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed17028-ab43-42d3-8431-0ebdc2c79d9b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:51.804323 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed17028-ab43-42d3-8431-0ebdc2c79d9b None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:51.805629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 63/133] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:28:51.810379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c87071-d25a-4ec8-aad7-0c33f1692258 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:51.810666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c87071-d25a-4ec8-aad7-0c33f1692258 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:51.810941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c87071-d25a-4ec8-aad7-0c33f1692258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:51.828942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-04c87071-d25a-4ec8-aad7-0c33f1692258 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:51.830121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-04c87071-d25a-4ec8-aad7-0c33f1692258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:51.879935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:51.890013 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c97cf-587f-48e1-b45f-1fd1f7bdd937 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:51.896366 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 64/134] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:28:51.923467 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45974127-e6aa-4a09-a161-ed9af734f533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:51.924078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45974127-e6aa-4a09-a161-ed9af734f533 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:51.924359 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45974127-e6aa-4a09-a161-ed9af734f533 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:51.924629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45974127-e6aa-4a09-a161-ed9af734f533 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:51.925271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45974127-e6aa-4a09-a161-ed9af734f533 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:51.926731 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45974127-e6aa-4a09-a161-ed9af734f533 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:51.949571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 71/135] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:28:51 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:52.182124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c87071-d25a-4ec8-aad7-0c33f1692258 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:52.246758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:52.291587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c87071-d25a-4ec8-aad7-0c33f1692258 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:52.330101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7944657-50ed-4fdd-b50b-d7e5eb929d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:52.331025 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7944657-50ed-4fdd-b50b-d7e5eb929d1f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:52.331300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7944657-50ed-4fdd-b50b-d7e5eb929d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:52.331568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7944657-50ed-4fdd-b50b-d7e5eb929d1f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:52.343611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7944657-50ed-4fdd-b50b-d7e5eb929d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-f7944657-50ed-4fdd-b50b-d7e5eb929d1f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:52.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:52.349317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:52.350101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 65/136] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:52.357472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45974127-e6aa-4a09-a161-ed9af734f533 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:52.465774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45974127-e6aa-4a09-a161-ed9af734f533 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:52.500723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a925d-6a2a-4581-9bb9-0ea32ce9f660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:52.501354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a925d-6a2a-4581-9bb9-0ea32ce9f660 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:52.501628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a925d-6a2a-4581-9bb9-0ea32ce9f660 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:52.501903 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a925d-6a2a-4581-9bb9-0ea32ce9f660 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:52.502601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a925d-6a2a-4581-9bb9-0ea32ce9f660 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:52.502902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a925d-6a2a-4581-9bb9-0ea32ce9f660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:52.504268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a925d-6a2a-4581-9bb9-0ea32ce9f660 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:52.513483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 72/137] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:28:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:52.519167 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7944657-50ed-4fdd-b50b-d7e5eb929d1f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:52.628701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7944657-50ed-4fdd-b50b-d7e5eb929d1f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:52.647327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:52.664776 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f49240-0bba-497f-b6c2-d0dccbfb6da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:52.665422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f49240-0bba-497f-b6c2-d0dccbfb6da7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:52.665705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f49240-0bba-497f-b6c2-d0dccbfb6da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:52.665977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f49240-0bba-497f-b6c2-d0dccbfb6da7 None admin] PATH_INFO: `/v3/users/fb194d5098de4c589de2e4386923b8a4` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:52.667416 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f49240-0bba-497f-b6c2-d0dccbfb6da7 None admin] RBAC: Authorizing `identity:get_user(user_id=fb194d5098de4c589de2e4386923b8a4)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:52.668881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f49240-0bba-497f-b6c2-d0dccbfb6da7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:52.670574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 66/138] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:52 2022] GET /identity/v3/users/fb194d5098de4c589de2e4386923b8a4 => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:52.676281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a925d-6a2a-4581-9bb9-0ea32ce9f660 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:52.690241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:52.785629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a925d-6a2a-4581-9bb9-0ea32ce9f660 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:52.824326 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3beb13-4e45-4580-8459-7eee560e10d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:52.824952 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3beb13-4e45-4580-8459-7eee560e10d5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:52.825237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3beb13-4e45-4580-8459-7eee560e10d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:52.825611 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3beb13-4e45-4580-8459-7eee560e10d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:52.896943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3beb13-4e45-4580-8459-7eee560e10d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:52.899066 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3beb13-4e45-4580-8459-7eee560e10d5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:52.911519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:53.011732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 73/139] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:28:53.013082 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f49240-0bba-497f-b6c2-d0dccbfb6da7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:53.122601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f49240-0bba-497f-b6c2-d0dccbfb6da7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:53.158126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771cb4f-e7a4-4eca-bf83-9b2de740366d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:53.158792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771cb4f-e7a4-4eca-bf83-9b2de740366d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:53.159074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771cb4f-e7a4-4eca-bf83-9b2de740366d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:53.159348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771cb4f-e7a4-4eca-bf83-9b2de740366d None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:53.160463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771cb4f-e7a4-4eca-bf83-9b2de740366d None admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:53.161914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771cb4f-e7a4-4eca-bf83-9b2de740366d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:53.163587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 67/140] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:53 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:53.169850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3beb13-4e45-4580-8459-7eee560e10d5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:53.274975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3beb13-4e45-4580-8459-7eee560e10d5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:53.287606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:53.315602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfa1ca-2129-410f-b42f-6e55338ece53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:53.316213 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfa1ca-2129-410f-b42f-6e55338ece53 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:53.316488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfa1ca-2129-410f-b42f-6e55338ece53 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:53.316808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfa1ca-2129-410f-b42f-6e55338ece53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:53.317529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfa1ca-2129-410f-b42f-6e55338ece53 None admin] RBAC: Adding query filter params (role.id=84fba578b2cd41e6958097aec20a6f6e, scope.project.id=a634485bda4247f4996f560aa4394790, user.id=fb194d5098de4c589de2e4386923b8a4) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:53.317830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfa1ca-2129-410f-b42f-6e55338ece53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:53.319367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfa1ca-2129-410f-b42f-6e55338ece53 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:53.325568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 74/141] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:28:53 2022] GET /identity/v3/role_assignments?user.id=fb194d5098de4c589de2e4386923b8a4&scope.project.id=a634485bda4247f4996f560aa4394790&role.id=84fba578b2cd41e6958097aec20a6f6e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:54.254177 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c699-a036-4499-9e8a-aaffba29ed94 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.254744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c699-a036-4499-9e8a-aaffba29ed94 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.254744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c699-a036-4499-9e8a-aaffba29ed94 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.256354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 68/142] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:54.262657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e14a4-f451-4551-b61c-420725f03ad9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.263182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e14a4-f451-4551-b61c-420725f03ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.263182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e14a4-f451-4551-b61c-420725f03ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.284252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3a3e14a4-f451-4551-b61c-420725f03ad9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:54.285464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3a3e14a4-f451-4551-b61c-420725f03ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:54.356521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 75/143] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:54.379032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf151e-da0e-4d40-8092-378470dc39a1 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.379319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf151e-da0e-4d40-8092-378470dc39a1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.379596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf151e-da0e-4d40-8092-378470dc39a1 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.381089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 69/144] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:54.386613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5d538-d58e-40c2-9a4b-6f67b0be2722 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.386902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5d538-d58e-40c2-9a4b-6f67b0be2722 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.387176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5d538-d58e-40c2-9a4b-6f67b0be2722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.405843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-37e5d538-d58e-40c2-9a4b-6f67b0be2722 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:54.407028 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-37e5d538-d58e-40c2-9a4b-6f67b0be2722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:54.476207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 76/145] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:54.579904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31131e-cafe-485a-8e3c-ddb55990f6f0 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.580207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31131e-cafe-485a-8e3c-ddb55990f6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.580466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31131e-cafe-485a-8e3c-ddb55990f6f0 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.581927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 70/146] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:54.587601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ff64-098a-4a30-8d4e-e18dbeb3a9cf None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.587893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ff64-098a-4a30-8d4e-e18dbeb3a9cf None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.588168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ff64-098a-4a30-8d4e-e18dbeb3a9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.606402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-68e4ff64-098a-4a30-8d4e-e18dbeb3a9cf None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:54.607577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-68e4ff64-098a-4a30-8d4e-e18dbeb3a9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:54.623573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31131e-cafe-485a-8e3c-ddb55990f6f0 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:54.639281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:54.682292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 77/147] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:54.689656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:54.737954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31131e-cafe-485a-8e3c-ddb55990f6f0 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:54.744241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4ff64-098a-4a30-8d4e-e18dbeb3a9cf None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:54.777786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f345077-3a81-401d-bdfa-5136ec629882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:54.778550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f345077-3a81-401d-bdfa-5136ec629882 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.778833 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f345077-3a81-401d-bdfa-5136ec629882 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.779106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f345077-3a81-401d-bdfa-5136ec629882 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.789584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f345077-3a81-401d-bdfa-5136ec629882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-4f345077-3a81-401d-bdfa-5136ec629882 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:54.795072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:54.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:54.796823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 71/148] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:28:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:28:54.801854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f345077-3a81-401d-bdfa-5136ec629882 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:54.853995 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4ff64-098a-4a30-8d4e-e18dbeb3a9cf None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:54.891222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364f881-d6a6-481b-9555-cee0e7e5e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:54.891863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364f881-d6a6-481b-9555-cee0e7e5e876 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.892208 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364f881-d6a6-481b-9555-cee0e7e5e876 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.892477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364f881-d6a6-481b-9555-cee0e7e5e876 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.903707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364f881-d6a6-481b-9555-cee0e7e5e876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:54.908425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f345077-3a81-401d-bdfa-5136ec629882 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-4364f881-d6a6-481b-9555-cee0e7e5e876 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:54.908556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:54.909411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:54.910232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 78/149] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:28:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:28:54.915927 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364f881-d6a6-481b-9555-cee0e7e5e876 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:54.927515 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:54.945257 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:54.951268 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b9583-6f46-44db-89ac-5c7389e43df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:54.952090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b9583-6f46-44db-89ac-5c7389e43df2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.952526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b9583-6f46-44db-89ac-5c7389e43df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.952951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b9583-6f46-44db-89ac-5c7389e43df2 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.953856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b9583-6f46-44db-89ac-5c7389e43df2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:54.954346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b9583-6f46-44db-89ac-5c7389e43df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:54.955950 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b9583-6f46-44db-89ac-5c7389e43df2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:54.965056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 72/150] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:54.965866 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b9583-6f46-44db-89ac-5c7389e43df2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.021044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364f881-d6a6-481b-9555-cee0e7e5e876 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.055635 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0b472-ae45-4757-a9ec-54958919cf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.056239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0b472-ae45-4757-a9ec-54958919cf80 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.056518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0b472-ae45-4757-a9ec-54958919cf80 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.056960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0b472-ae45-4757-a9ec-54958919cf80 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.057626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0b472-ae45-4757-a9ec-54958919cf80 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:55.057930 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0b472-ae45-4757-a9ec-54958919cf80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.059338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0b472-ae45-4757-a9ec-54958919cf80 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.067017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 79/151] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:28:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:28:55.067162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b9583-6f46-44db-89ac-5c7389e43df2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.067791 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0b472-ae45-4757-a9ec-54958919cf80 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.101642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b0976-f591-443d-8293-661fd126d0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.102299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b0976-f591-443d-8293-661fd126d0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.102578 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b0976-f591-443d-8293-661fd126d0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.102846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b0976-f591-443d-8293-661fd126d0b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.103935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b0976-f591-443d-8293-661fd126d0b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.105369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b0976-f591-443d-8293-661fd126d0b6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.106961 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 73/152] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:55.107711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b0976-f591-443d-8293-661fd126d0b6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.152753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:55.154599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:55.170214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0b472-ae45-4757-a9ec-54958919cf80 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.204557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3707b7-c0ea-4b57-8190-0b4a48419905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.205191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3707b7-c0ea-4b57-8190-0b4a48419905 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.205474 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3707b7-c0ea-4b57-8190-0b4a48419905 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.205741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3707b7-c0ea-4b57-8190-0b4a48419905 None admin] PATH_INFO: `/v3/users/f3f2d0d990ca4c27b26f02d4167f94c1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.211485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b0976-f591-443d-8293-661fd126d0b6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.219493 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3707b7-c0ea-4b57-8190-0b4a48419905 None admin] RBAC: Authorizing `identity:get_user(user_id=f3f2d0d990ca4c27b26f02d4167f94c1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.220962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3707b7-c0ea-4b57-8190-0b4a48419905 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.222621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 80/153] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:55 2022] GET /identity/v3/users/f3f2d0d990ca4c27b26f02d4167f94c1 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:55.223364 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3707b7-c0ea-4b57-8190-0b4a48419905 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.250121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc292b-90ca-4860-b40f-3bd9341bae0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.250769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc292b-90ca-4860-b40f-3bd9341bae0d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.251052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc292b-90ca-4860-b40f-3bd9341bae0d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.251322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc292b-90ca-4860-b40f-3bd9341bae0d None admin] PATH_INFO: `/v3/users/f3f2d0d990ca4c27b26f02d4167f94c1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.252436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc292b-90ca-4860-b40f-3bd9341bae0d None admin] RBAC: Authorizing `identity:get_user(user_id=f3f2d0d990ca4c27b26f02d4167f94c1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.253860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc292b-90ca-4860-b40f-3bd9341bae0d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.255616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 74/154] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:55 2022] GET /identity/v3/users/f3f2d0d990ca4c27b26f02d4167f94c1 => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:55.256280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc292b-90ca-4860-b40f-3bd9341bae0d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.323204 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3707b7-c0ea-4b57-8190-0b4a48419905 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.353622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc292b-90ca-4860-b40f-3bd9341bae0d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.356293 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba07096-5405-4305-ae40-fc40331f7dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.356900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba07096-5405-4305-ae40-fc40331f7dad None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.357176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba07096-5405-4305-ae40-fc40331f7dad None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.357451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba07096-5405-4305-ae40-fc40331f7dad None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.358097 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba07096-5405-4305-ae40-fc40331f7dad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.359540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba07096-5405-4305-ae40-fc40331f7dad None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.367095 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:55.376742 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:55.390147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5f573-9fa1-4044-8717-7aed83d692d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.390890 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5f573-9fa1-4044-8717-7aed83d692d5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.391194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5f573-9fa1-4044-8717-7aed83d692d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.391469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5f573-9fa1-4044-8717-7aed83d692d5 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.392757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5f573-9fa1-4044-8717-7aed83d692d5 None admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.394280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5f573-9fa1-4044-8717-7aed83d692d5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.396228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 75/155] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:55 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:55.397105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5f573-9fa1-4044-8717-7aed83d692d5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.398027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 81/156] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:28:55 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:55.402909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba07096-5405-4305-ae40-fc40331f7dad None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.515673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5f573-9fa1-4044-8717-7aed83d692d5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.516291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba07096-5405-4305-ae40-fc40331f7dad None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.556101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d50d5-0a2d-4f39-9879-9ed552646cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.556330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4d188-1e2c-4cb1-9479-0e63d2351499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.556751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d50d5-0a2d-4f39-9879-9ed552646cfc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.557013 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4d188-1e2c-4cb1-9479-0e63d2351499 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.557101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d50d5-0a2d-4f39-9879-9ed552646cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.557171 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4d188-1e2c-4cb1-9479-0e63d2351499 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.557286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d50d5-0a2d-4f39-9879-9ed552646cfc None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.557385 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4d188-1e2c-4cb1-9479-0e63d2351499 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555/users/f3f2d0d990ca4c27b26f02d4167f94c1/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.558440 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d50d5-0a2d-4f39-9879-9ed552646cfc None admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.559287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4d188-1e2c-4cb1-9479-0e63d2351499 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d5203564be545e1a1d8a33a58625555, user_id=f3f2d0d990ca4c27b26f02d4167f94c1, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.559895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d50d5-0a2d-4f39-9879-9ed552646cfc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.560712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4d188-1e2c-4cb1-9479-0e63d2351499 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.561545 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 76/157] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:55 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:55.567330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d50d5-0a2d-4f39-9879-9ed552646cfc None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.577263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 82/158] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:28:55 2022] PUT /identity/v3/projects/7d5203564be545e1a1d8a33a58625555/users/f3f2d0d990ca4c27b26f02d4167f94c1/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:55.639331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:55.708190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d50d5-0a2d-4f39-9879-9ed552646cfc None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.753858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02604956-03fe-476a-81d0-b2b694921f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.754571 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604956-03fe-476a-81d0-b2b694921f58 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.754846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604956-03fe-476a-81d0-b2b694921f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.755192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604956-03fe-476a-81d0-b2b694921f58 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555/users/f3f2d0d990ca4c27b26f02d4167f94c1/roles/f948e6f6ee4f42de92195aee35b05641` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.757752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02604956-03fe-476a-81d0-b2b694921f58 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d5203564be545e1a1d8a33a58625555, user_id=f3f2d0d990ca4c27b26f02d4167f94c1, role_id=f948e6f6ee4f42de92195aee35b05641)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.759228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02604956-03fe-476a-81d0-b2b694921f58 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.777507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 77/159] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:28:55 2022] PUT /identity/v3/projects/7d5203564be545e1a1d8a33a58625555/users/f3f2d0d990ca4c27b26f02d4167f94c1/roles/f948e6f6ee4f42de92195aee35b05641 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:57.815563 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36129800-83f0-46e8-8849-83bf4fd5cf8d None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:57.816080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36129800-83f0-46e8-8849-83bf4fd5cf8d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:57.816176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36129800-83f0-46e8-8849-83bf4fd5cf8d None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:57.817631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 83/160] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:57.823523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb658070-b584-413c-897a-1c34686736c6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:57.824032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb658070-b584-413c-897a-1c34686736c6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:57.824032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb658070-b584-413c-897a-1c34686736c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:57.827267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457fa4a-df43-4c58-8569-4e81e0b70a04 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:57.827856 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457fa4a-df43-4c58-8569-4e81e0b70a04 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:57.828382 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457fa4a-df43-4c58-8569-4e81e0b70a04 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:57.830242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 84/161] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:57.835919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600583c9-9142-4880-9342-975f5a508596 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:57.836199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600583c9-9142-4880-9342-975f5a508596 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:57.836473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600583c9-9142-4880-9342-975f5a508596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:57.846136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-eb658070-b584-413c-897a-1c34686736c6 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:57.847643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-eb658070-b584-413c-897a-1c34686736c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:57.854949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-600583c9-9142-4880-9342-975f5a508596 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:57.856109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-600583c9-9142-4880-9342-975f5a508596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:57.919932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:57.930493 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 78/162] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:57.936591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 85/163] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:58.053098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c84c3-89a2-4dd5-8008-7b04b42629e6 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.053828 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c84c3-89a2-4dd5-8008-7b04b42629e6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.054340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c84c3-89a2-4dd5-8008-7b04b42629e6 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.056031 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 79/164] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:28:58.062363 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6baa09-9dac-4c28-98e6-8f4fa4a5554d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.062973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6baa09-9dac-4c28-98e6-8f4fa4a5554d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.063460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6baa09-9dac-4c28-98e6-8f4fa4a5554d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.082607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0d6baa09-9dac-4c28-98e6-8f4fa4a5554d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:58.084073 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0d6baa09-9dac-4c28-98e6-8f4fa4a5554d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:58.157768 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784488a-e5ae-43b8-ad97-f21789225386 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.158114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784488a-e5ae-43b8-ad97-f21789225386 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.158364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784488a-e5ae-43b8-ad97-f21789225386 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.158674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 86/165] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:58.162913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 80/166] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:28:58.165148 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8322472-3dac-4a87-aa8c-9df9bba9b4fc None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.165442 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8322472-3dac-4a87-aa8c-9df9bba9b4fc None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.165718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8322472-3dac-4a87-aa8c-9df9bba9b4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.183971 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e8322472-3dac-4a87-aa8c-9df9bba9b4fc None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:58.185144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e8322472-3dac-4a87-aa8c-9df9bba9b4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:58.201280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784488a-e5ae-43b8-ad97-f21789225386 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:58.244568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:58.266807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 87/167] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:28:58.270761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8322472-3dac-4a87-aa8c-9df9bba9b4fc None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:58.315455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784488a-e5ae-43b8-ad97-f21789225386 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:58.335863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:58.355816 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666e2fa-964f-4886-82eb-73929ec2859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:58.356601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666e2fa-964f-4886-82eb-73929ec2859d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.357238 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666e2fa-964f-4886-82eb-73929ec2859d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.357238 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666e2fa-964f-4886-82eb-73929ec2859d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.367869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666e2fa-964f-4886-82eb-73929ec2859d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-0666e2fa-964f-4886-82eb-73929ec2859d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:58.372760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:58.373620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:58.374422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 81/168] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:28:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:28:58.381048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666e2fa-964f-4886-82eb-73929ec2859d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:58.406429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8322472-3dac-4a87-aa8c-9df9bba9b4fc None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:58.451456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac807c-e5c5-490b-8777-f68e672e4e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:58.452090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac807c-e5c5-490b-8777-f68e672e4e7d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.452369 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac807c-e5c5-490b-8777-f68e672e4e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.452639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac807c-e5c5-490b-8777-f68e672e4e7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.463386 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac807c-e5c5-490b-8777-f68e672e4e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-2cac807c-e5c5-490b-8777-f68e672e4e7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:58.468192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:58.469069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:28:58.469875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 88/169] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:58.470593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac807c-e5c5-490b-8777-f68e672e4e7d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:58.487279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:58.492345 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666e2fa-964f-4886-82eb-73929ec2859d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:58.529843 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a2a18-6cf4-4b7b-abbf-9a47292f2819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:58.530480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a2a18-6cf4-4b7b-abbf-9a47292f2819 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.530750 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a2a18-6cf4-4b7b-abbf-9a47292f2819 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.531013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a2a18-6cf4-4b7b-abbf-9a47292f2819 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.531664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a2a18-6cf4-4b7b-abbf-9a47292f2819 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:58.531957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a2a18-6cf4-4b7b-abbf-9a47292f2819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:58.533326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a2a18-6cf4-4b7b-abbf-9a47292f2819 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:58.542418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 82/170] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:28:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:58.543439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a2a18-6cf4-4b7b-abbf-9a47292f2819 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:58.584324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac807c-e5c5-490b-8777-f68e672e4e7d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:58.595473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:58.619086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e836f63-a892-42e9-be4b-b6ab2a50f28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:58.619711 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e836f63-a892-42e9-be4b-b6ab2a50f28b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.619990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e836f63-a892-42e9-be4b-b6ab2a50f28b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.620257 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e836f63-a892-42e9-be4b-b6ab2a50f28b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.621360 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e836f63-a892-42e9-be4b-b6ab2a50f28b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:58.622832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e836f63-a892-42e9-be4b-b6ab2a50f28b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:58.624401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 89/171] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:28:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:58.624401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e836f63-a892-42e9-be4b-b6ab2a50f28b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:58.645758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a2a18-6cf4-4b7b-abbf-9a47292f2819 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:58.679330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616286e-1f50-4cf0-8690-3a7d3bfe1cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:58.679932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616286e-1f50-4cf0-8690-3a7d3bfe1cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.680215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616286e-1f50-4cf0-8690-3a7d3bfe1cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.680483 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616286e-1f50-4cf0-8690-3a7d3bfe1cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.681141 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616286e-1f50-4cf0-8690-3a7d3bfe1cb9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:58.681441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616286e-1f50-4cf0-8690-3a7d3bfe1cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:58.682836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616286e-1f50-4cf0-8690-3a7d3bfe1cb9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:58.690910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 83/172] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:28:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:58.696146 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616286e-1f50-4cf0-8690-3a7d3bfe1cb9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:58.718338 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:58.726575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e836f63-a892-42e9-be4b-b6ab2a50f28b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:58.760544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc431498-c3ba-40f7-9e48-d44fe91b8de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:58.761244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431498-c3ba-40f7-9e48-d44fe91b8de9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.761523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431498-c3ba-40f7-9e48-d44fe91b8de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.761785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431498-c3ba-40f7-9e48-d44fe91b8de9 None admin] PATH_INFO: `/v3/users/f3f2d0d990ca4c27b26f02d4167f94c1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.762907 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc431498-c3ba-40f7-9e48-d44fe91b8de9 None admin] RBAC: Authorizing `identity:get_user(user_id=f3f2d0d990ca4c27b26f02d4167f94c1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:58.764345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc431498-c3ba-40f7-9e48-d44fe91b8de9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:58.765907 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 90/173] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:58 2022] GET /identity/v3/users/f3f2d0d990ca4c27b26f02d4167f94c1 => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:58.765907 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc431498-c3ba-40f7-9e48-d44fe91b8de9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:58.793063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616286e-1f50-4cf0-8690-3a7d3bfe1cb9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:58.816643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:58.826483 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40fa3c-9b6d-4647-b5af-f713e18cffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:58.827154 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40fa3c-9b6d-4647-b5af-f713e18cffb0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.827422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40fa3c-9b6d-4647-b5af-f713e18cffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.827694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40fa3c-9b6d-4647-b5af-f713e18cffb0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.837889 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40fa3c-9b6d-4647-b5af-f713e18cffb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-ce40fa3c-9b6d-4647-b5af-f713e18cffb0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:28:58.843233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:28:58.844280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 84/174] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:28:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:28:58.845064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40fa3c-9b6d-4647-b5af-f713e18cffb0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:58.873163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc431498-c3ba-40f7-9e48-d44fe91b8de9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:58.906376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14013a-f93d-4201-a602-7ff5a8d1d19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:58.906972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14013a-f93d-4201-a602-7ff5a8d1d19b None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.907245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14013a-f93d-4201-a602-7ff5a8d1d19b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.907512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14013a-f93d-4201-a602-7ff5a8d1d19b None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:58.908152 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14013a-f93d-4201-a602-7ff5a8d1d19b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:58.909578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14013a-f93d-4201-a602-7ff5a8d1d19b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:58.926630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:58.931486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 91/175] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:28:58 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:58.932303 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14013a-f93d-4201-a602-7ff5a8d1d19b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:58.956580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40fa3c-9b6d-4647-b5af-f713e18cffb0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:58.997939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd662e-f560-4768-94ec-7525f1a9ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:58.998629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd662e-f560-4768-94ec-7525f1a9ae14 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:58.998905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd662e-f560-4768-94ec-7525f1a9ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:58.999174 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd662e-f560-4768-94ec-7525f1a9ae14 None admin] PATH_INFO: `/v3/users/071e5325cfd84442a6a77aea46e3ca0a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:59.017507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd662e-f560-4768-94ec-7525f1a9ae14 None admin] RBAC: Authorizing `identity:get_user(user_id=071e5325cfd84442a6a77aea46e3ca0a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:59.018983 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd662e-f560-4768-94ec-7525f1a9ae14 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:59.025507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 85/176] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:58 2022] GET /identity/v3/users/071e5325cfd84442a6a77aea46e3ca0a => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:59.025507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd662e-f560-4768-94ec-7525f1a9ae14 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:59.051959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14013a-f93d-4201-a602-7ff5a8d1d19b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:59.054412 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:59.091100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d09b2-da0a-4dac-b96e-12fcfa4a5429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:59.091732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d09b2-da0a-4dac-b96e-12fcfa4a5429 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:59.092017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d09b2-da0a-4dac-b96e-12fcfa4a5429 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:59.092288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d09b2-da0a-4dac-b96e-12fcfa4a5429 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:59.140266 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd662e-f560-4768-94ec-7525f1a9ae14 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:59.169301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d09b2-da0a-4dac-b96e-12fcfa4a5429 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:59.170835 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d09b2-da0a-4dac-b96e-12fcfa4a5429 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:59.184293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4a771-2a3f-44c2-8cd1-a0c5889364ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:59.184908 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4a771-2a3f-44c2-8cd1-a0c5889364ce None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:59.186016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4a771-2a3f-44c2-8cd1-a0c5889364ce None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:59.186016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4a771-2a3f-44c2-8cd1-a0c5889364ce None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:59.186329 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4a771-2a3f-44c2-8cd1-a0c5889364ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:59.186622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4a771-2a3f-44c2-8cd1-a0c5889364ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:59.189219 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4a771-2a3f-44c2-8cd1-a0c5889364ce None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:59.200278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 86/177] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:28:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:59.200278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4a771-2a3f-44c2-8cd1-a0c5889364ce None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:59.202896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:59.311928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 92/178] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:28:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:28:59.313805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d09b2-da0a-4dac-b96e-12fcfa4a5429 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:59.316533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:59.337195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4a771-2a3f-44c2-8cd1-a0c5889364ce None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:59.382507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7a496-e015-4177-a199-99526d87eaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:59.383151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7a496-e015-4177-a199-99526d87eaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:59.383429 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7a496-e015-4177-a199-99526d87eaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:59.383700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7a496-e015-4177-a199-99526d87eaa5 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:59.385090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7a496-e015-4177-a199-99526d87eaa5 None admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:59.387324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7a496-e015-4177-a199-99526d87eaa5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:59.389179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 87/179] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:59 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:59.390008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7a496-e015-4177-a199-99526d87eaa5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:59.433520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d09b2-da0a-4dac-b96e-12fcfa4a5429 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:59.472680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:59.484124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636949d-d933-49c6-8524-110da8fee8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:59.484778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636949d-d933-49c6-8524-110da8fee8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:59.485058 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636949d-d933-49c6-8524-110da8fee8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:59.485326 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636949d-d933-49c6-8524-110da8fee8a0 None admin] PATH_INFO: `/v3/users/f3f2d0d990ca4c27b26f02d4167f94c1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:59.486888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636949d-d933-49c6-8524-110da8fee8a0 None admin] RBAC: Authorizing `identity:get_user(user_id=f3f2d0d990ca4c27b26f02d4167f94c1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:59.488678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636949d-d933-49c6-8524-110da8fee8a0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:59.491541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 93/180] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:28:59 2022] GET /identity/v3/users/f3f2d0d990ca4c27b26f02d4167f94c1 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:59.492469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636949d-d933-49c6-8524-110da8fee8a0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:59.514365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7a496-e015-4177-a199-99526d87eaa5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:59.561882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e018cab-3a32-4651-86c7-e0deae989f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:59.562541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e018cab-3a32-4651-86c7-e0deae989f20 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:59.562822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e018cab-3a32-4651-86c7-e0deae989f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:59.563087 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e018cab-3a32-4651-86c7-e0deae989f20 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:59.564197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e018cab-3a32-4651-86c7-e0deae989f20 None admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:59.565922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e018cab-3a32-4651-86c7-e0deae989f20 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:59.569369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 88/181] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:59 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:59.570224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e018cab-3a32-4651-86c7-e0deae989f20 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:59.570942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:59.613187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636949d-d933-49c6-8524-110da8fee8a0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:59.660855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a392d-eda6-4fc2-be13-166d1b392246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:59.661944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a392d-eda6-4fc2-be13-166d1b392246 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:59.662250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a392d-eda6-4fc2-be13-166d1b392246 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:59.662526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a392d-eda6-4fc2-be13-166d1b392246 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555/users/071e5325cfd84442a6a77aea46e3ca0a/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:59.665281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a392d-eda6-4fc2-be13-166d1b392246 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d5203564be545e1a1d8a33a58625555, user_id=071e5325cfd84442a6a77aea46e3ca0a, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:59.667754 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a392d-eda6-4fc2-be13-166d1b392246 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:59.687256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 94/182] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:28:59 2022] PUT /identity/v3/projects/7d5203564be545e1a1d8a33a58625555/users/071e5325cfd84442a6a77aea46e3ca0a/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:59.687897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e018cab-3a32-4651-86c7-e0deae989f20 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:59.688006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a392d-eda6-4fc2-be13-166d1b392246 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:59.731773 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837e561-a137-4fe2-a13b-418562dab6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:59.732432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837e561-a137-4fe2-a13b-418562dab6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:59.732972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837e561-a137-4fe2-a13b-418562dab6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:59.733247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837e561-a137-4fe2-a13b-418562dab6d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:59.747518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:59.819761 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837e561-a137-4fe2-a13b-418562dab6d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:59.821571 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837e561-a137-4fe2-a13b-418562dab6d9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:59.834185 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a392d-eda6-4fc2-be13-166d1b392246 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:59.878986 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:59.887799 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c9fba-3fdf-4123-bbed-c8c0e3760caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:59.888994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c9fba-3fdf-4123-bbed-c8c0e3760caa None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:59.889880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c9fba-3fdf-4123-bbed-c8c0e3760caa None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:59.890656 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c9fba-3fdf-4123-bbed-c8c0e3760caa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:59.891940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c9fba-3fdf-4123-bbed-c8c0e3760caa None admin] RBAC: Adding query filter params (role.id=8b32d66a3cb3483c8ede361ad8637f66, scope.project.id=7d5203564be545e1a1d8a33a58625555, user.id=f3f2d0d990ca4c27b26f02d4167f94c1) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:59.892916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c9fba-3fdf-4123-bbed-c8c0e3760caa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:59.894985 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c9fba-3fdf-4123-bbed-c8c0e3760caa None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:59.904947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 95/183] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:28:59 2022] GET /identity/v3/role_assignments?user.id=f3f2d0d990ca4c27b26f02d4167f94c1&scope.project.id=7d5203564be545e1a1d8a33a58625555&role.id=8b32d66a3cb3483c8ede361ad8637f66 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:59.942657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 89/184] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:28:59.948076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c9fba-3fdf-4123-bbed-c8c0e3760caa None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:00.076030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c9fba-3fdf-4123-bbed-c8c0e3760caa None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:00.115300 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff727de0-a9c8-4d4f-90ff-3f58ccf75ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:00.116841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727de0-a9c8-4d4f-90ff-3f58ccf75ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:00.117347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727de0-a9c8-4d4f-90ff-3f58ccf75ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:00.117821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727de0-a9c8-4d4f-90ff-3f58ccf75ce8 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:00.119323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff727de0-a9c8-4d4f-90ff-3f58ccf75ce8 None admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:00.121068 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff727de0-a9c8-4d4f-90ff-3f58ccf75ce8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:00.123087 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 96/185] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:28:59 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:00.129113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837e561-a137-4fe2-a13b-418562dab6d9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:00.184370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:00.256276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837e561-a137-4fe2-a13b-418562dab6d9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:00.295661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa5068-169f-42c8-9edc-88603e2329e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:00.296279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa5068-169f-42c8-9edc-88603e2329e0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:00.296561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa5068-169f-42c8-9edc-88603e2329e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:00.296832 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa5068-169f-42c8-9edc-88603e2329e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:00.297559 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa5068-169f-42c8-9edc-88603e2329e0 None admin] RBAC: Adding query filter params (role.id=f948e6f6ee4f42de92195aee35b05641, scope.project.id=7d5203564be545e1a1d8a33a58625555, user.id=f3f2d0d990ca4c27b26f02d4167f94c1) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:00.297865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa5068-169f-42c8-9edc-88603e2329e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:00.299339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa5068-169f-42c8-9edc-88603e2329e0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:00.305885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 90/186] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:29:00 2022] GET /identity/v3/role_assignments?user.id=f3f2d0d990ca4c27b26f02d4167f94c1&scope.project.id=7d5203564be545e1a1d8a33a58625555&role.id=f948e6f6ee4f42de92195aee35b05641 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:01.401201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f97df-d4f5-49de-b3e5-80163de3a052 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:01.401735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f97df-d4f5-49de-b3e5-80163de3a052 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:01.401735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f97df-d4f5-49de-b3e5-80163de3a052 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:01.403412 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 97/187] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:01.408932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fa2eb-ab9e-405e-98c5-8971b953aad3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:01.409422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fa2eb-ab9e-405e-98c5-8971b953aad3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:01.409523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fa2eb-ab9e-405e-98c5-8971b953aad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:01.429001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9d3fa2eb-ab9e-405e-98c5-8971b953aad3 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:01.430266 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9d3fa2eb-ab9e-405e-98c5-8971b953aad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:01.500337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 91/188] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:01.531790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5821bbc-1d91-4012-a170-f54d4a4b27de None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:01.532106 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5821bbc-1d91-4012-a170-f54d4a4b27de None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:01.532342 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5821bbc-1d91-4012-a170-f54d4a4b27de None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:01.533812 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 98/189] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:01.538913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c696a71-f2be-45f4-9550-4956c8afa589 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:01.539203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c696a71-f2be-45f4-9550-4956c8afa589 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:01.539473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c696a71-f2be-45f4-9550-4956c8afa589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:01.558285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6c696a71-f2be-45f4-9550-4956c8afa589 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:01.559450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6c696a71-f2be-45f4-9550-4956c8afa589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:01.585019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:01.626859 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 92/190] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:01.770221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5821bbc-1d91-4012-a170-f54d4a4b27de None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:01.833594 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:01.874522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5821bbc-1d91-4012-a170-f54d4a4b27de None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:01.893379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c696a71-f2be-45f4-9550-4956c8afa589 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:01.910455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb1e59-529c-4f82-a255-7bd57db6a66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:01.911261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb1e59-529c-4f82-a255-7bd57db6a66f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:01.911539 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb1e59-529c-4f82-a255-7bd57db6a66f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:01.912102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb1e59-529c-4f82-a255-7bd57db6a66f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:01.926319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb1e59-529c-4f82-a255-7bd57db6a66f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-1edb1e59-529c-4f82-a255-7bd57db6a66f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:01.933351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:01.934252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:01.935048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 99/191] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:01.940899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb1e59-529c-4f82-a255-7bd57db6a66f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:01.997493 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c696a71-f2be-45f4-9550-4956c8afa589 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.036298 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd4dce-33da-4753-a8ec-4a560d794e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.037310 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd4dce-33da-4753-a8ec-4a560d794e21 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.038412 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd4dce-33da-4753-a8ec-4a560d794e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.038687 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd4dce-33da-4753-a8ec-4a560d794e21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.039924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd4dce-33da-4753-a8ec-4a560d794e21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.042507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd4dce-33da-4753-a8ec-4a560d794e21 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.044822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 93/192] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:02.048140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb1e59-529c-4f82-a255-7bd57db6a66f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.052022 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd4dce-33da-4753-a8ec-4a560d794e21 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.076007 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:02.086885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:02.088194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44089e40-36cd-464b-b7b6-80033bf9f16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.088875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44089e40-36cd-464b-b7b6-80033bf9f16e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.089127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44089e40-36cd-464b-b7b6-80033bf9f16e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.089420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44089e40-36cd-464b-b7b6-80033bf9f16e None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.090117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44089e40-36cd-464b-b7b6-80033bf9f16e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:02.090447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44089e40-36cd-464b-b7b6-80033bf9f16e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.091819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44089e40-36cd-464b-b7b6-80033bf9f16e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.100586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 100/193] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:29:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:02.106219 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44089e40-36cd-464b-b7b6-80033bf9f16e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.165175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd4dce-33da-4753-a8ec-4a560d794e21 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.199425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f186890-87c9-4808-bea0-6e161ccc700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.200039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f186890-87c9-4808-bea0-6e161ccc700d None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.200314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f186890-87c9-4808-bea0-6e161ccc700d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.200581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f186890-87c9-4808-bea0-6e161ccc700d None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.201231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f186890-87c9-4808-bea0-6e161ccc700d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.202699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f186890-87c9-4808-bea0-6e161ccc700d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.213828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44089e40-36cd-464b-b7b6-80033bf9f16e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.232620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 94/194] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:29:02 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:29:02.239876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a262865-40b8-4f4a-a263-a1346c78ec03 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.240507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a262865-40b8-4f4a-a263-a1346c78ec03 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.240849 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a262865-40b8-4f4a-a263-a1346c78ec03 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.242286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 95/195] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:02.248476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ecbb5-e96b-4aff-ba75-f87ac57da3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.249144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ecbb5-e96b-4aff-ba75-f87ac57da3fb None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.249419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ecbb5-e96b-4aff-ba75-f87ac57da3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.249689 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ecbb5-e96b-4aff-ba75-f87ac57da3fb None admin] PATH_INFO: `/v3/users/5ad445a5c0e3487b844db186249e2f53` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.252382 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde41e0-e8b8-4e9d-be40-0023ee2a4ba8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.252714 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde41e0-e8b8-4e9d-be40-0023ee2a4ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.252990 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde41e0-e8b8-4e9d-be40-0023ee2a4ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.266056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ecbb5-e96b-4aff-ba75-f87ac57da3fb None admin] RBAC: Authorizing `identity:get_user(user_id=5ad445a5c0e3487b844db186249e2f53)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.267572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ecbb5-e96b-4aff-ba75-f87ac57da3fb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.269307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 101/196] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:29:02 2022] GET /identity/v3/users/5ad445a5c0e3487b844db186249e2f53 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:02.275523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8cde41e0-e8b8-4e9d-be40-0023ee2a4ba8 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:02.275703 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ecbb5-e96b-4aff-ba75-f87ac57da3fb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.276792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8cde41e0-e8b8-4e9d-be40-0023ee2a4ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:02.324475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:02.362365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 96/197] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:29:02.389733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ecbb5-e96b-4aff-ba75-f87ac57da3fb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.429666 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce9b6-001e-4af8-961c-382760dd1f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.430322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce9b6-001e-4af8-961c-382760dd1f9b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.430602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce9b6-001e-4af8-961c-382760dd1f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.430863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ce9b6-001e-4af8-961c-382760dd1f9b None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.431978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ce9b6-001e-4af8-961c-382760dd1f9b None admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.433460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ce9b6-001e-4af8-961c-382760dd1f9b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.435351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 102/198] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:29:02 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:02.440829 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde41e0-e8b8-4e9d-be40-0023ee2a4ba8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.459900 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:02.555037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde41e0-e8b8-4e9d-be40-0023ee2a4ba8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.592683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dee728-cfe3-4316-8da1-6bcfce68fd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.593267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dee728-cfe3-4316-8da1-6bcfce68fd81 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.593541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dee728-cfe3-4316-8da1-6bcfce68fd81 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.593811 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dee728-cfe3-4316-8da1-6bcfce68fd81 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555/users/5ad445a5c0e3487b844db186249e2f53/roles/84fba578b2cd41e6958097aec20a6f6e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.595730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dee728-cfe3-4316-8da1-6bcfce68fd81 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d5203564be545e1a1d8a33a58625555, user_id=5ad445a5c0e3487b844db186249e2f53, role_id=84fba578b2cd41e6958097aec20a6f6e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.597548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dee728-cfe3-4316-8da1-6bcfce68fd81 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.612662 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 97/199] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:29:02 2022] PUT /identity/v3/projects/7d5203564be545e1a1d8a33a58625555/users/5ad445a5c0e3487b844db186249e2f53/roles/84fba578b2cd41e6958097aec20a6f6e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:02.632245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce9b6-001e-4af8-961c-382760dd1f9b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.751177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ce9b6-001e-4af8-961c-382760dd1f9b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.766575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:02.792782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe17acd-e027-4187-9d86-bdf1ca91cc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.793438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe17acd-e027-4187-9d86-bdf1ca91cc08 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.793700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe17acd-e027-4187-9d86-bdf1ca91cc08 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.793973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe17acd-e027-4187-9d86-bdf1ca91cc08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.805294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe17acd-e027-4187-9d86-bdf1ca91cc08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-dbe17acd-e027-4187-9d86-bdf1ca91cc08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:02.810159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:02.811023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:02.811790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 103/200] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:02.817291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dee728-cfe3-4316-8da1-6bcfce68fd81 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.902989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:02.936025 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dee728-cfe3-4316-8da1-6bcfce68fd81 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.971925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3dc046-0c37-4347-b036-d3d9dab43a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.972535 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dc046-0c37-4347-b036-d3d9dab43a08 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.972810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dc046-0c37-4347-b036-d3d9dab43a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.973077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dc046-0c37-4347-b036-d3d9dab43a08 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.973732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3dc046-0c37-4347-b036-d3d9dab43a08 None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:02.974027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3dc046-0c37-4347-b036-d3d9dab43a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.975464 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3dc046-0c37-4347-b036-d3d9dab43a08 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.984326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 98/201] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:29:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:02.989915 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe17acd-e027-4187-9d86-bdf1ca91cc08 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:03.095766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe17acd-e027-4187-9d86-bdf1ca91cc08 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:03.135911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0523da0-52c4-4925-8bec-66eefd2b78c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:03.136556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0523da0-52c4-4925-8bec-66eefd2b78c3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:03.136832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0523da0-52c4-4925-8bec-66eefd2b78c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:03.137099 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0523da0-52c4-4925-8bec-66eefd2b78c3 None admin] PATH_INFO: `/v3/users/071e5325cfd84442a6a77aea46e3ca0a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:03.138242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0523da0-52c4-4925-8bec-66eefd2b78c3 None admin] RBAC: Authorizing `identity:get_user(user_id=071e5325cfd84442a6a77aea46e3ca0a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:03.139662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0523da0-52c4-4925-8bec-66eefd2b78c3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:03.141232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 104/202] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:29:02 2022] GET /identity/v3/users/071e5325cfd84442a6a77aea46e3ca0a => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:03.146960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3dc046-0c37-4347-b036-d3d9dab43a08 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:03.263442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3dc046-0c37-4347-b036-d3d9dab43a08 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:03.298559 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfa304-ac2a-41e9-9460-3fd397293749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:03.299169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfa304-ac2a-41e9-9460-3fd397293749 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:03.299446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfa304-ac2a-41e9-9460-3fd397293749 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:03.299713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfa304-ac2a-41e9-9460-3fd397293749 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:03.308711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:03.373000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfa304-ac2a-41e9-9460-3fd397293749 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:03.374524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfa304-ac2a-41e9-9460-3fd397293749 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:03.477404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 99/203] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:29:03.483016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0523da0-52c4-4925-8bec-66eefd2b78c3 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:03.524039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:03.593672 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0523da0-52c4-4925-8bec-66eefd2b78c3 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:03.627347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6142e86-b982-45ff-8e52-f9cbef140b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:03.627979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6142e86-b982-45ff-8e52-f9cbef140b06 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:03.628256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6142e86-b982-45ff-8e52-f9cbef140b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:03.628525 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6142e86-b982-45ff-8e52-f9cbef140b06 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:03.629642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6142e86-b982-45ff-8e52-f9cbef140b06 None admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:03.631611 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6142e86-b982-45ff-8e52-f9cbef140b06 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:03.633372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 105/204] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:29:03 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:03.639406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfa304-ac2a-41e9-9460-3fd397293749 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:03.680188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:03.743820 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfa304-ac2a-41e9-9460-3fd397293749 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:03.779077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3a8d7-c12b-4c72-bd79-420b70571b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:03.779686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3a8d7-c12b-4c72-bd79-420b70571b1a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:03.779965 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3a8d7-c12b-4c72-bd79-420b70571b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:03.780232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3a8d7-c12b-4c72-bd79-420b70571b1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:03.780953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3a8d7-c12b-4c72-bd79-420b70571b1a None admin] RBAC: Adding query filter params (role.id=09a932be0c1b481c8494705d2030fc2a, scope.project.id=7d5203564be545e1a1d8a33a58625555, user.id=071e5325cfd84442a6a77aea46e3ca0a) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:03.781259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3a8d7-c12b-4c72-bd79-420b70571b1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:03.782928 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3a8d7-c12b-4c72-bd79-420b70571b1a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:03.789187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 100/205] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:29:03 2022] GET /identity/v3/role_assignments?user.id=071e5325cfd84442a6a77aea46e3ca0a&scope.project.id=7d5203564be545e1a1d8a33a58625555&role.id=09a932be0c1b481c8494705d2030fc2a => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:04.669316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed78f7-4742-4c8c-babe-af59425a2535 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:04.669829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed78f7-4742-4c8c-babe-af59425a2535 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:04.669829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed78f7-4742-4c8c-babe-af59425a2535 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:04.671391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 106/206] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:04.676376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d2993-ec7d-4746-a163-02c647f30120 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:04.676765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d2993-ec7d-4746-a163-02c647f30120 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:04.676922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d2993-ec7d-4746-a163-02c647f30120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:04.697087 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e99c6-8d13-4451-9543-e4353954e4cb None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:04.697364 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e99c6-8d13-4451-9543-e4353954e4cb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:04.697495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a07d2993-ec7d-4746-a163-02c647f30120 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:04.697905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e99c6-8d13-4451-9543-e4353954e4cb None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:04.698595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a07d2993-ec7d-4746-a163-02c647f30120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:04.699568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 107/207] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:29:04.704516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8305d-dd32-40cf-a3ae-f5c688166002 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:04.704953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8305d-dd32-40cf-a3ae-f5c688166002 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:04.705367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8305d-dd32-40cf-a3ae-f5c688166002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:04.723354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-2be8305d-dd32-40cf-a3ae-f5c688166002 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:04.724725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2be8305d-dd32-40cf-a3ae-f5c688166002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:04.770421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 101/208] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:04.807599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 108/209] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:29:04.876154 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe143f-da1b-43f2-8324-d379029f8323 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:04.876458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe143f-da1b-43f2-8324-d379029f8323 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:04.876716 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe143f-da1b-43f2-8324-d379029f8323 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:04.878230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 102/210] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:04.883421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5884b7-4235-44f3-8050-c92bb99743ea None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:04.883715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5884b7-4235-44f3-8050-c92bb99743ea None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:04.883992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5884b7-4235-44f3-8050-c92bb99743ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:04.902301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-8d5884b7-4235-44f3-8050-c92bb99743ea None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:04.903474 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8d5884b7-4235-44f3-8050-c92bb99743ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:04.934424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:04.973821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 109/211] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:05.045757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe143f-da1b-43f2-8324-d379029f8323 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:05.077601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:05.077860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5884b7-4235-44f3-8050-c92bb99743ea None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:05.151752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe143f-da1b-43f2-8324-d379029f8323 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:05.186460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5884b7-4235-44f3-8050-c92bb99743ea None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:05.187304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fd71c-4afb-45d9-b6d6-e8c742514033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:05.187935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fd71c-4afb-45d9-b6d6-e8c742514033 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:05.188214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fd71c-4afb-45d9-b6d6-e8c742514033 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:05.188483 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fd71c-4afb-45d9-b6d6-e8c742514033 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:05.201218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fd71c-4afb-45d9-b6d6-e8c742514033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-2f5fd71c-4afb-45d9-b6d6-e8c742514033 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:05.206379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:05.207286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:05.213561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 103/212] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:05.214828 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fd71c-4afb-45d9-b6d6-e8c742514033 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:05.234753 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05f2d4-c659-404b-b63b-d0cfb91b4612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:05.235445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05f2d4-c659-404b-b63b-d0cfb91b4612 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:05.235727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05f2d4-c659-404b-b63b-d0cfb91b4612 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:05.235998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05f2d4-c659-404b-b63b-d0cfb91b4612 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:05.237084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05f2d4-c659-404b-b63b-d0cfb91b4612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:05.238544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05f2d4-c659-404b-b63b-d0cfb91b4612 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:05.240100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 110/213] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:05.243450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05f2d4-c659-404b-b63b-d0cfb91b4612 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:05.276899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:05.309227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:05.318620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fd71c-4afb-45d9-b6d6-e8c742514033 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:05.342308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05f2d4-c659-404b-b63b-d0cfb91b4612 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:05.352613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f407ca7-be70-403c-86b4-367a2a57ccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:05.353208 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f407ca7-be70-403c-86b4-367a2a57ccd3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:05.353481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f407ca7-be70-403c-86b4-367a2a57ccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:05.353750 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f407ca7-be70-403c-86b4-367a2a57ccd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:05.354421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f407ca7-be70-403c-86b4-367a2a57ccd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:05.354717 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f407ca7-be70-403c-86b4-367a2a57ccd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:05.356096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f407ca7-be70-403c-86b4-367a2a57ccd3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:05.363887 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 104/214] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:29:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:05.364678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f407ca7-be70-403c-86b4-367a2a57ccd3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:05.381548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ac7a3-90a9-4dbc-b837-db3c1fc2a3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:05.382488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ac7a3-90a9-4dbc-b837-db3c1fc2a3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:05.382770 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ac7a3-90a9-4dbc-b837-db3c1fc2a3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:05.383039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ac7a3-90a9-4dbc-b837-db3c1fc2a3f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:05.393273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ac7a3-90a9-4dbc-b837-db3c1fc2a3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f88ac7a3-90a9-4dbc-b837-db3c1fc2a3f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:05.398916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:05.399811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 111/215] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:05.405674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ac7a3-90a9-4dbc-b837-db3c1fc2a3f2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:05.465394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f407ca7-be70-403c-86b4-367a2a57ccd3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:05.495745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:05.499002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c2603-d999-48af-9d41-7670430a0434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:05.499607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c2603-d999-48af-9d41-7670430a0434 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:05.500064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c2603-d999-48af-9d41-7670430a0434 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:05.500185 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c2603-d999-48af-9d41-7670430a0434 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:05.500827 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c2603-d999-48af-9d41-7670430a0434 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:05.502312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c2603-d999-48af-9d41-7670430a0434 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:05.502541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ac7a3-90a9-4dbc-b837-db3c1fc2a3f2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:05.523426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 105/216] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:29:05 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:05.524241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c2603-d999-48af-9d41-7670430a0434 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:05.537069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d7a1c-1060-4d10-af2e-9810f58ab998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:05.538368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d7a1c-1060-4d10-af2e-9810f58ab998 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:05.538657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d7a1c-1060-4d10-af2e-9810f58ab998 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:05.538932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d7a1c-1060-4d10-af2e-9810f58ab998 None admin] PATH_INFO: `/v3/users/6d02bdef40a3400e96dd98f6a3ed7ba2` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:05.539560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:05.556515 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d7a1c-1060-4d10-af2e-9810f58ab998 None admin] RBAC: Authorizing `identity:get_user(user_id=6d02bdef40a3400e96dd98f6a3ed7ba2)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:05.558018 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d7a1c-1060-4d10-af2e-9810f58ab998 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:05.559788 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 112/217] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:29:05 2022] GET /identity/v3/users/6d02bdef40a3400e96dd98f6a3ed7ba2 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:05.565803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d7a1c-1060-4d10-af2e-9810f58ab998 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:05.651147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c2603-d999-48af-9d41-7670430a0434 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:05.681514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d7a1c-1060-4d10-af2e-9810f58ab998 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:05.692958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2c6c0-93d4-4a11-b7cb-32370a8e9fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:05.694376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2c6c0-93d4-4a11-b7cb-32370a8e9fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:05.694797 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2c6c0-93d4-4a11-b7cb-32370a8e9fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:05.695075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2c6c0-93d4-4a11-b7cb-32370a8e9fd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:05.695754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2c6c0-93d4-4a11-b7cb-32370a8e9fd1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:05.696055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2c6c0-93d4-4a11-b7cb-32370a8e9fd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:05.697455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2c6c0-93d4-4a11-b7cb-32370a8e9fd1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:05.711217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 106/218] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:29:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:05.717102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2c6c0-93d4-4a11-b7cb-32370a8e9fd1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:05.726600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d44b7b-de6a-4af0-8ea3-e99527ffde0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:05.727215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d44b7b-de6a-4af0-8ea3-e99527ffde0c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:05.727482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d44b7b-de6a-4af0-8ea3-e99527ffde0c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:05.727753 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d44b7b-de6a-4af0-8ea3-e99527ffde0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:05.728468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d44b7b-de6a-4af0-8ea3-e99527ffde0c None admin] RBAC: Adding query filter params (role.id=09a932be0c1b481c8494705d2030fc2a, user.id=6d02bdef40a3400e96dd98f6a3ed7ba2, scope.system=all) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:05.728770 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d44b7b-de6a-4af0-8ea3-e99527ffde0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:05.730264 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d44b7b-de6a-4af0-8ea3-e99527ffde0c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:05.735365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 113/219] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Oct 24 13:29:05 2022] GET /identity/v3/role_assignments?user.id=6d02bdef40a3400e96dd98f6a3ed7ba2&scope.system=all&role.id=09a932be0c1b481c8494705d2030fc2a => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:05.800537 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:05.839116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2c6c0-93d4-4a11-b7cb-32370a8e9fd1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:05.880310 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c7dcd-ea23-4e09-a589-421e46077257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:05.881325 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c7dcd-ea23-4e09-a589-421e46077257 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:05.881791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c7dcd-ea23-4e09-a589-421e46077257 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:05.882059 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c7dcd-ea23-4e09-a589-421e46077257 None admin] PATH_INFO: `/v3/users/5ad445a5c0e3487b844db186249e2f53` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:05.883751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c7dcd-ea23-4e09-a589-421e46077257 None admin] RBAC: Authorizing `identity:get_user(user_id=5ad445a5c0e3487b844db186249e2f53)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:05.885220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c7dcd-ea23-4e09-a589-421e46077257 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:05.886901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 107/220] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:29:05 2022] GET /identity/v3/users/5ad445a5c0e3487b844db186249e2f53 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:05.892890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d44b7b-de6a-4af0-8ea3-e99527ffde0c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:05.914083 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:06.008463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d44b7b-de6a-4af0-8ea3-e99527ffde0c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:06.045029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b10de-4933-4036-a48d-5b6d3d4d1fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:06.045647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b10de-4933-4036-a48d-5b6d3d4d1fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:06.045958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b10de-4933-4036-a48d-5b6d3d4d1fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:06.046230 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b10de-4933-4036-a48d-5b6d3d4d1fa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:06.117763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b10de-4933-4036-a48d-5b6d3d4d1fa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:06.119607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b10de-4933-4036-a48d-5b6d3d4d1fa4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:06.138638 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:06.234287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 114/221] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:29:06.242189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c7dcd-ea23-4e09-a589-421e46077257 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:06.352586 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c7dcd-ea23-4e09-a589-421e46077257 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:06.387301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71195a2-d158-4189-8b83-17bb88d36af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:06.387934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71195a2-d158-4189-8b83-17bb88d36af1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:06.388216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71195a2-d158-4189-8b83-17bb88d36af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:06.388487 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71195a2-d158-4189-8b83-17bb88d36af1 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:06.389608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71195a2-d158-4189-8b83-17bb88d36af1 None admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:06.391232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71195a2-d158-4189-8b83-17bb88d36af1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:06.393294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 108/222] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:29:06 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:06.399513 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b10de-4933-4036-a48d-5b6d3d4d1fa4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:06.504638 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b10de-4933-4036-a48d-5b6d3d4d1fa4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:06.524214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:06.543838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781920a0-21f3-4ad9-8a38-5f577c580f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:06.544775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781920a0-21f3-4ad9-8a38-5f577c580f72 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:06.545246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781920a0-21f3-4ad9-8a38-5f577c580f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:06.545707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781920a0-21f3-4ad9-8a38-5f577c580f72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:06.546685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781920a0-21f3-4ad9-8a38-5f577c580f72 None admin] RBAC: Adding query filter params (role.id=84fba578b2cd41e6958097aec20a6f6e, scope.project.id=7d5203564be545e1a1d8a33a58625555, user.id=5ad445a5c0e3487b844db186249e2f53) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:06.547209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781920a0-21f3-4ad9-8a38-5f577c580f72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:06.548847 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781920a0-21f3-4ad9-8a38-5f577c580f72 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:06.562315 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 115/223] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:29:06 2022] GET /identity/v3/role_assignments?user.id=5ad445a5c0e3487b844db186249e2f53&scope.project.id=7d5203564be545e1a1d8a33a58625555&role.id=84fba578b2cd41e6958097aec20a6f6e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:07.900828 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92efc-ff32-432c-8622-d417a401377a None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:07.901377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92efc-ff32-432c-8622-d417a401377a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:07.901377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92efc-ff32-432c-8622-d417a401377a None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:07.902792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 109/224] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:07.907925 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5faf5-fb76-41f9-a8eb-f826387e58b9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:07.908301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5faf5-fb76-41f9-a8eb-f826387e58b9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:07.908472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5faf5-fb76-41f9-a8eb-f826387e58b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:07.926769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-28c5faf5-fb76-41f9-a8eb-f826387e58b9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:07.928064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-28c5faf5-fb76-41f9-a8eb-f826387e58b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:07.965038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4d76c-b2a7-4eff-b72e-5e3e7ec717a2 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:07.965317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4d76c-b2a7-4eff-b72e-5e3e7ec717a2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:07.965589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4d76c-b2a7-4eff-b72e-5e3e7ec717a2 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:07.966977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 110/225] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:29:07.971405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f45064-6e5d-43b8-a610-9b39ba2139e4 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:07.971688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f45064-6e5d-43b8-a610-9b39ba2139e4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:07.971964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f45064-6e5d-43b8-a610-9b39ba2139e4 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:07.973219 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 111/226] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:29:07.974197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf423c-3d4c-4708-871a-dd960b5ce458 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:07.974481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf423c-3d4c-4708-871a-dd960b5ce458 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:07.974780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf423c-3d4c-4708-871a-dd960b5ce458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:07.992420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bfcf423c-3d4c-4708-871a-dd960b5ce458 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:07.993611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bfcf423c-3d4c-4708-871a-dd960b5ce458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:07.995778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:07.999145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 116/227] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:08.007160 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b575416-2b81-455f-bbb7-066f41df78cc None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:08.007160 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b575416-2b81-455f-bbb7-066f41df78cc None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:08.007160 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b575416-2b81-455f-bbb7-066f41df78cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:08.019157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3b575416-2b81-455f-bbb7-066f41df78cc None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:08.020334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3b575416-2b81-455f-bbb7-066f41df78cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:08.073291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 112/228] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:08.092745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 117/229] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:08.276796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf423c-3d4c-4708-871a-dd960b5ce458 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:08.342026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b575416-2b81-455f-bbb7-066f41df78cc None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:08.384209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf423c-3d4c-4708-871a-dd960b5ce458 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:08.384396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:08.418312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe792319-1272-47ca-b499-fd57b7fe9aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:08.418941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe792319-1272-47ca-b499-fd57b7fe9aef None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:08.419223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe792319-1272-47ca-b499-fd57b7fe9aef None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:08.419495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe792319-1272-47ca-b499-fd57b7fe9aef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:08.426495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:08.432094 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe792319-1272-47ca-b499-fd57b7fe9aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-fe792319-1272-47ca-b499-fd57b7fe9aef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:08.436925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:08.437985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 113/230] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:08.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe792319-1272-47ca-b499-fd57b7fe9aef None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:08.448906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b575416-2b81-455f-bbb7-066f41df78cc None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:08.482831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c25a04-692f-4fb9-b11a-614c26bbac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:08.483403 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c25a04-692f-4fb9-b11a-614c26bbac9d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:08.483685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c25a04-692f-4fb9-b11a-614c26bbac9d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:08.483958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c25a04-692f-4fb9-b11a-614c26bbac9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:08.485053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c25a04-692f-4fb9-b11a-614c26bbac9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:08.486577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c25a04-692f-4fb9-b11a-614c26bbac9d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:08.488288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 118/231] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:08.488855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c25a04-692f-4fb9-b11a-614c26bbac9d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:08.536283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe792319-1272-47ca-b499-fd57b7fe9aef None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:08.570939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a648d-ea6f-48d2-82eb-2bced1cc6179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:08.571587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a648d-ea6f-48d2-82eb-2bced1cc6179 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:08.571869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a648d-ea6f-48d2-82eb-2bced1cc6179 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:08.572136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a648d-ea6f-48d2-82eb-2bced1cc6179 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:08.581996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a648d-ea6f-48d2-82eb-2bced1cc6179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-d15a648d-ea6f-48d2-82eb-2bced1cc6179 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:08.586867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:08.587786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:08.588596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 114/232] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:08.589260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a648d-ea6f-48d2-82eb-2bced1cc6179 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:08.589325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c25a04-692f-4fb9-b11a-614c26bbac9d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:08.601646 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:08.627576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc05be4-5898-4451-b092-4f3caada6362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:08.628184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc05be4-5898-4451-b092-4f3caada6362 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:08.628462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc05be4-5898-4451-b092-4f3caada6362 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:08.628733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc05be4-5898-4451-b092-4f3caada6362 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:08.629389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc05be4-5898-4451-b092-4f3caada6362 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:08.629690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc05be4-5898-4451-b092-4f3caada6362 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:08.631089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc05be4-5898-4451-b092-4f3caada6362 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:08.638911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 119/233] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:29:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:08.639771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc05be4-5898-4451-b092-4f3caada6362 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:08.653988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:08.697387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a648d-ea6f-48d2-82eb-2bced1cc6179 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:08.731977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bd9a5-1365-4ebb-8ed4-55c7100e829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:08.732582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd9a5-1365-4ebb-8ed4-55c7100e829a None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:08.732906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd9a5-1365-4ebb-8ed4-55c7100e829a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:08.733138 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd9a5-1365-4ebb-8ed4-55c7100e829a None admin] PATH_INFO: `/v3/groups` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:08.733753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971bd9a5-1365-4ebb-8ed4-55c7100e829a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:08.735228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971bd9a5-1365-4ebb-8ed4-55c7100e829a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:08.736599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc05be4-5898-4451-b092-4f3caada6362 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:08.752668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 115/234] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Oct 24 13:29:08 2022] POST /identity/v3/groups => generated 229 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:08.753383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bd9a5-1365-4ebb-8ed4-55c7100e829a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:08.776053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25a095-6f11-40de-b4b9-ce918edbcd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:08.776681 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25a095-6f11-40de-b4b9-ce918edbcd18 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:08.776956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25a095-6f11-40de-b4b9-ce918edbcd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:08.777226 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25a095-6f11-40de-b4b9-ce918edbcd18 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:08.777902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25a095-6f11-40de-b4b9-ce918edbcd18 None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:08.778228 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25a095-6f11-40de-b4b9-ce918edbcd18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:08.779686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25a095-6f11-40de-b4b9-ce918edbcd18 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:08.790586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 120/235] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:29:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:08.796862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25a095-6f11-40de-b4b9-ce918edbcd18 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:08.838338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:08.888637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bd9a5-1365-4ebb-8ed4-55c7100e829a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:08.909339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:08.920224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25a095-6f11-40de-b4b9-ce918edbcd18 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:08.931572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f5ef8-01f5-4921-a162-54f0c6bd8329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:08.932602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f5ef8-01f5-4921-a162-54f0c6bd8329 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:08.932602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f5ef8-01f5-4921-a162-54f0c6bd8329 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:08.932848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f5ef8-01f5-4921-a162-54f0c6bd8329 None admin] PATH_INFO: `/v3/users/d3c0844c4a224dcca9e1aafec42f249e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:08.949642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f5ef8-01f5-4921-a162-54f0c6bd8329 None admin] RBAC: Authorizing `identity:get_user(user_id=d3c0844c4a224dcca9e1aafec42f249e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:08.951496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f5ef8-01f5-4921-a162-54f0c6bd8329 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:08.953082 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 116/236] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:29:08 2022] GET /identity/v3/users/d3c0844c4a224dcca9e1aafec42f249e => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:08.958732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f5ef8-01f5-4921-a162-54f0c6bd8329 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:08.960819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcce5bb-fa12-4e7c-92ad-35a42926dc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:08.961466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcce5bb-fa12-4e7c-92ad-35a42926dc8d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:08.961750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcce5bb-fa12-4e7c-92ad-35a42926dc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:08.962149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcce5bb-fa12-4e7c-92ad-35a42926dc8d None admin] PATH_INFO: `/v3/users/6d02bdef40a3400e96dd98f6a3ed7ba2` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:08.963326 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcce5bb-fa12-4e7c-92ad-35a42926dc8d None admin] RBAC: Authorizing `identity:get_user(user_id=6d02bdef40a3400e96dd98f6a3ed7ba2)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:08.964842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcce5bb-fa12-4e7c-92ad-35a42926dc8d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:08.966592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 121/237] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:29:08 2022] GET /identity/v3/users/6d02bdef40a3400e96dd98f6a3ed7ba2 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:08.973884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcce5bb-fa12-4e7c-92ad-35a42926dc8d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:09.069962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f5ef8-01f5-4921-a162-54f0c6bd8329 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:09.086535 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:09.090073 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcce5bb-fa12-4e7c-92ad-35a42926dc8d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:09.106817 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a72eb9-0cb3-410a-a5f8-947bf2009674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:09.107435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a72eb9-0cb3-410a-a5f8-947bf2009674 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:09.107713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a72eb9-0cb3-410a-a5f8-947bf2009674 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:09.107981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a72eb9-0cb3-410a-a5f8-947bf2009674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:09.108702 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a72eb9-0cb3-410a-a5f8-947bf2009674 None admin] RBAC: Adding query filter params (role.id=84fba578b2cd41e6958097aec20a6f6e, user.id=d3c0844c4a224dcca9e1aafec42f249e, scope.system=all) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:09.109007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a72eb9-0cb3-410a-a5f8-947bf2009674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:09.110455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a72eb9-0cb3-410a-a5f8-947bf2009674 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:09.116228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 117/238] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Oct 24 13:29:08 2022] GET /identity/v3/role_assignments?user.id=d3c0844c4a224dcca9e1aafec42f249e&scope.system=all&role.id=84fba578b2cd41e6958097aec20a6f6e => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:09.125128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8e0f7-8be8-4c12-a46b-fed159a30df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:09.125820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8e0f7-8be8-4c12-a46b-fed159a30df4 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:09.126104 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8e0f7-8be8-4c12-a46b-fed159a30df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:09.126412 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8e0f7-8be8-4c12-a46b-fed159a30df4 None admin] PATH_INFO: `/v3/system/users/6d02bdef40a3400e96dd98f6a3ed7ba2/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:09.127930 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8e0f7-8be8-4c12-a46b-fed159a30df4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6d02bdef40a3400e96dd98f6a3ed7ba2, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:09.129393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8e0f7-8be8-4c12-a46b-fed159a30df4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:09.140327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 122/239] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Oct 24 13:29:08 2022] PUT /identity/v3/system/users/6d02bdef40a3400e96dd98f6a3ed7ba2/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:29:11.005235 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973cd4-3f5b-4dd8-b6eb-bd0bd8f938b1 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:11.005753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973cd4-3f5b-4dd8-b6eb-bd0bd8f938b1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:11.005827 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973cd4-3f5b-4dd8-b6eb-bd0bd8f938b1 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:11.007386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 118/240] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:11.012668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a6b5-59e7-4387-bbbf-5c70ceae6d09 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:11.013111 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a6b5-59e7-4387-bbbf-5c70ceae6d09 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:11.013252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a6b5-59e7-4387-bbbf-5c70ceae6d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:11.032616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9114a6b5-59e7-4387-bbbf-5c70ceae6d09 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:11.033803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9114a6b5-59e7-4387-bbbf-5c70ceae6d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:11.105653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 123/241] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:11.284657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20cdd3-4ec2-4fb0-bbff-3c9b34d9ae16 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:11.284954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20cdd3-4ec2-4fb0-bbff-3c9b34d9ae16 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:11.285119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20cdd3-4ec2-4fb0-bbff-3c9b34d9ae16 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:11.285333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd3fc4-08b5-4859-b71e-da5490c7f6aa None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:11.285620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd3fc4-08b5-4859-b71e-da5490c7f6aa None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:11.285893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd3fc4-08b5-4859-b71e-da5490c7f6aa None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:11.290812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 119/242] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:11.290942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 124/243] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:11.292571 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35628338-6d6a-4e9d-a1da-6a45dbd27c94 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:11.292852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35628338-6d6a-4e9d-a1da-6a45dbd27c94 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:11.292948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d7d04-012f-401d-9516-40f26f9ced3c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:11.293120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35628338-6d6a-4e9d-a1da-6a45dbd27c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:11.293225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d7d04-012f-401d-9516-40f26f9ced3c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:11.293379 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d7d04-012f-401d-9516-40f26f9ced3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:11.311258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-758d7d04-012f-401d-9516-40f26f9ced3c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:11.312430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-758d7d04-012f-401d-9516-40f26f9ced3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:11.313008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-35628338-6d6a-4e9d-a1da-6a45dbd27c94 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:11.314159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-35628338-6d6a-4e9d-a1da-6a45dbd27c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:11.353194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:11.388201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 125/244] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:29:11.389043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d7d04-012f-401d-9516-40f26f9ced3c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:11.394995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 120/245] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:29:11.417514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:11.492913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d7d04-012f-401d-9516-40f26f9ced3c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:11.526579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360a768-e9c6-4574-8554-fec2103d4291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:11.527205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360a768-e9c6-4574-8554-fec2103d4291 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:11.527480 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360a768-e9c6-4574-8554-fec2103d4291 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:11.527747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360a768-e9c6-4574-8554-fec2103d4291 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:11.528821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360a768-e9c6-4574-8554-fec2103d4291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:11.530268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360a768-e9c6-4574-8554-fec2103d4291 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:11.531971 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 126/246] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:11.537885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35628338-6d6a-4e9d-a1da-6a45dbd27c94 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:11.634058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35628338-6d6a-4e9d-a1da-6a45dbd27c94 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:11.664347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360a768-e9c6-4574-8554-fec2103d4291 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:11.667606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326109c-9b28-4f18-80e5-aa396ac8f6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:11.668397 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326109c-9b28-4f18-80e5-aa396ac8f6d9 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:11.668813 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326109c-9b28-4f18-80e5-aa396ac8f6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:11.669221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326109c-9b28-4f18-80e5-aa396ac8f6d9 None admin] PATH_INFO: `/v3/groups` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:11.669988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326109c-9b28-4f18-80e5-aa396ac8f6d9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:11.671598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326109c-9b28-4f18-80e5-aa396ac8f6d9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:11.686369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 121/247] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Oct 24 13:29:11 2022] POST /identity/v3/groups => generated 226 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:11.695582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326109c-9b28-4f18-80e5-aa396ac8f6d9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:11.732705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:11.777374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:11.788552 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360a768-e9c6-4574-8554-fec2103d4291 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:11.821436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326109c-9b28-4f18-80e5-aa396ac8f6d9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:11.833966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e3c84-4f81-487f-b0eb-d77f16f84ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:11.835540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e3c84-4f81-487f-b0eb-d77f16f84ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:11.835905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e3c84-4f81-487f-b0eb-d77f16f84ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:11.836579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e3c84-4f81-487f-b0eb-d77f16f84ad0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:11.848706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e3c84-4f81-487f-b0eb-d77f16f84ad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-542e3c84-4f81-487f-b0eb-d77f16f84ad0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:11.854209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:11.855121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:11.855920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 127/248] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:11.861021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e3c84-4f81-487f-b0eb-d77f16f84ad0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:11.869188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b100769-1ec5-4e65-be13-e85778e45809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:11.869736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b100769-1ec5-4e65-be13-e85778e45809 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:11.870544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b100769-1ec5-4e65-be13-e85778e45809 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:11.870729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b100769-1ec5-4e65-be13-e85778e45809 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:11.883622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b100769-1ec5-4e65-be13-e85778e45809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6b100769-1ec5-4e65-be13-e85778e45809 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:11.889675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:11.890638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:11.891414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 122/249] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:11.897726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b100769-1ec5-4e65-be13-e85778e45809 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:11.982461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e3c84-4f81-487f-b0eb-d77f16f84ad0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:12.020847 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1f33f-f4b6-4727-acbe-509958bb0198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:12.021480 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1f33f-f4b6-4727-acbe-509958bb0198 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:12.021880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1f33f-f4b6-4727-acbe-509958bb0198 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:12.022874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1f33f-f4b6-4727-acbe-509958bb0198 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:12.023188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1f33f-f4b6-4727-acbe-509958bb0198 None admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:12.023491 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1f33f-f4b6-4727-acbe-509958bb0198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:12.024895 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1f33f-f4b6-4727-acbe-509958bb0198 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:12.026476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:12.035776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 128/250] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:29:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:12.041443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b100769-1ec5-4e65-be13-e85778e45809 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:12.043513 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1f33f-f4b6-4727-acbe-509958bb0198 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:12.061031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:12.092107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30edaf-f65c-4891-89bd-9e3f0b674ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:12.093027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30edaf-f65c-4891-89bd-9e3f0b674ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:12.093027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30edaf-f65c-4891-89bd-9e3f0b674ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:12.093027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30edaf-f65c-4891-89bd-9e3f0b674ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:12.093669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30edaf-f65c-4891-89bd-9e3f0b674ad7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:12.093969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30edaf-f65c-4891-89bd-9e3f0b674ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:12.096392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30edaf-f65c-4891-89bd-9e3f0b674ad7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:12.108370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 123/251] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:29:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:12.113891 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30edaf-f65c-4891-89bd-9e3f0b674ad7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:12.175243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1f33f-f4b6-4727-acbe-509958bb0198 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:12.232119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57ccdc-0bba-4a36-93ef-f6ccd96909c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:12.232768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57ccdc-0bba-4a36-93ef-f6ccd96909c6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:12.233043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57ccdc-0bba-4a36-93ef-f6ccd96909c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:12.233309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57ccdc-0bba-4a36-93ef-f6ccd96909c6 None admin] PATH_INFO: `/v3/users/6d02bdef40a3400e96dd98f6a3ed7ba2` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:12.234864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57ccdc-0bba-4a36-93ef-f6ccd96909c6 None admin] RBAC: Authorizing `identity:get_user(user_id=6d02bdef40a3400e96dd98f6a3ed7ba2)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:12.237225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57ccdc-0bba-4a36-93ef-f6ccd96909c6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:12.239945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 129/252] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:29:12 2022] GET /identity/v3/users/6d02bdef40a3400e96dd98f6a3ed7ba2 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:12.247725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57ccdc-0bba-4a36-93ef-f6ccd96909c6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:12.260858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30edaf-f65c-4891-89bd-9e3f0b674ad7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:12.309653 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cc761-a0eb-4471-9bba-4595fcf13796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:12.310681 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cc761-a0eb-4471-9bba-4595fcf13796 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:12.311156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cc761-a0eb-4471-9bba-4595fcf13796 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:12.311615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cc761-a0eb-4471-9bba-4595fcf13796 None admin] PATH_INFO: `/v3/users/d3c0844c4a224dcca9e1aafec42f249e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:12.313798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cc761-a0eb-4471-9bba-4595fcf13796 None admin] RBAC: Authorizing `identity:get_user(user_id=d3c0844c4a224dcca9e1aafec42f249e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:12.315716 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cc761-a0eb-4471-9bba-4595fcf13796 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:12.318074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 124/253] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:29:12 2022] GET /identity/v3/users/d3c0844c4a224dcca9e1aafec42f249e => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:12.324173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cc761-a0eb-4471-9bba-4595fcf13796 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:12.346060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:12.362616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:12.368616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57ccdc-0bba-4a36-93ef-f6ccd96909c6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:12.406191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ed38d-5277-404a-960f-200b132631db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:12.406955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ed38d-5277-404a-960f-200b132631db None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:12.407607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ed38d-5277-404a-960f-200b132631db None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:12.407607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ed38d-5277-404a-960f-200b132631db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:12.408362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ed38d-5277-404a-960f-200b132631db None admin] RBAC: Adding query filter params (role.id=09a932be0c1b481c8494705d2030fc2a, user.id=6d02bdef40a3400e96dd98f6a3ed7ba2, scope.system=all) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:12.408726 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ed38d-5277-404a-960f-200b132631db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:12.410275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ed38d-5277-404a-960f-200b132631db None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:12.426249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 130/254] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Oct 24 13:29:12 2022] GET /identity/v3/role_assignments?user.id=6d02bdef40a3400e96dd98f6a3ed7ba2&scope.system=all&role.id=09a932be0c1b481c8494705d2030fc2a => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:12.448683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cc761-a0eb-4471-9bba-4595fcf13796 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:12.487226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f694f1b-6287-4b09-a6e0-dbaf6ca25a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:12.488219 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f694f1b-6287-4b09-a6e0-dbaf6ca25a7b None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:12.488671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f694f1b-6287-4b09-a6e0-dbaf6ca25a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:12.489115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f694f1b-6287-4b09-a6e0-dbaf6ca25a7b None admin] PATH_INFO: `/v3/system/users/d3c0844c4a224dcca9e1aafec42f249e/roles/84fba578b2cd41e6958097aec20a6f6e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:12.491121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f694f1b-6287-4b09-a6e0-dbaf6ca25a7b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d3c0844c4a224dcca9e1aafec42f249e, role_id=84fba578b2cd41e6958097aec20a6f6e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:12.492784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f694f1b-6287-4b09-a6e0-dbaf6ca25a7b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:12.504021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 125/255] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Oct 24 13:29:12 2022] PUT /identity/v3/system/users/d3c0844c4a224dcca9e1aafec42f249e/roles/84fba578b2cd41e6958097aec20a6f6e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:14.237696 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0130189-5c5f-45b3-82ec-d10153718b21 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:14.238314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0130189-5c5f-45b3-82ec-d10153718b21 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:14.238314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0130189-5c5f-45b3-82ec-d10153718b21 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:14.239757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 131/256] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:14.245898 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92d397-2a5d-4ec5-834d-3c966efd9e9f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:14.246359 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92d397-2a5d-4ec5-834d-3c966efd9e9f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:14.246499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92d397-2a5d-4ec5-834d-3c966efd9e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:14.273089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-5c92d397-2a5d-4ec5-834d-3c966efd9e9f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:14.274394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5c92d397-2a5d-4ec5-834d-3c966efd9e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:14.346700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 126/257] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 13:29:14.631532 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4182c-44bc-4e56-99da-2312e125d788 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:14.632056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4182c-44bc-4e56-99da-2312e125d788 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:14.632333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4182c-44bc-4e56-99da-2312e125d788 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:14.633957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 132/258] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:14.640406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b628f8e-54a2-48e8-8b3d-a12991d0ad63 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:14.640696 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b628f8e-54a2-48e8-8b3d-a12991d0ad63 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:14.640971 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b628f8e-54a2-48e8-8b3d-a12991d0ad63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:14.664463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8b628f8e-54a2-48e8-8b3d-a12991d0ad63 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:14.665668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8b628f8e-54a2-48e8-8b3d-a12991d0ad63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:14.677481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 127/259] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 13:29:14.889867 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4182c-44bc-4e56-99da-2312e125d788 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:14.908266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4182c-44bc-4e56-99da-2312e125d788 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:14.912956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7bb06-367a-48ed-850d-e15509efe112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:14.913805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7bb06-367a-48ed-850d-e15509efe112 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:14.914311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7bb06-367a-48ed-850d-e15509efe112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:14.914775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7bb06-367a-48ed-850d-e15509efe112 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:14.930646 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7bb06-367a-48ed-850d-e15509efe112 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-7ba7bb06-367a-48ed-850d-e15509efe112 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:14.935838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:14.936680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:14.937477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:14.939719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 133/260] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Oct 24 13:29:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:14.946404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b628f8e-54a2-48e8-8b3d-a12991d0ad63 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:14.960032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b628f8e-54a2-48e8-8b3d-a12991d0ad63 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:14.970853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01481403-fcf3-4c0d-9a55-3364b99ec47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:14.971476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01481403-fcf3-4c0d-9a55-3364b99ec47e admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:14.971757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01481403-fcf3-4c0d-9a55-3364b99ec47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:14.972027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01481403-fcf3-4c0d-9a55-3364b99ec47e admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:14.972701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01481403-fcf3-4c0d-9a55-3364b99ec47e admin admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:14.972998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01481403-fcf3-4c0d-9a55-3364b99ec47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:14.975023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-01481403-fcf3-4c0d-9a55-3364b99ec47e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:29:14.975023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Oct 24 13:29:14.975023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: Oct 24 13:29:14.976166 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01481403-fcf3-4c0d-9a55-3364b99ec47e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:14.981147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7bb06-367a-48ed-850d-e15509efe112 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:14.993365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 128/261] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:29:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:14.993707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7bb06-367a-48ed-850d-e15509efe112 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:14.998495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199fdb-30cd-48e7-867b-477bacc703d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:14.998627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01481403-fcf3-4c0d-9a55-3364b99ec47e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:14.999574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199fdb-30cd-48e7-867b-477bacc703d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.000045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199fdb-30cd-48e7-867b-477bacc703d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.000520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199fdb-30cd-48e7-867b-477bacc703d3 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.011572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01481403-fcf3-4c0d-9a55-3364b99ec47e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:15.014688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8199fdb-30cd-48e7-867b-477bacc703d3 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-c8199fdb-30cd-48e7-867b-477bacc703d3 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:15.020324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:15.021181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:29:15.022021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:15.024459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005083db-fbbf-4435-8650-0e211d578aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:15.025107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005083db-fbbf-4435-8650-0e211d578aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.025207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 134/262] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Oct 24 13:29:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:15.025388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005083db-fbbf-4435-8650-0e211d578aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.027815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005083db-fbbf-4435-8650-0e211d578aa2 admin admin] PATH_INFO: `/v3/groups/80584f1064a74e0a8b82e2dd91025db0` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.030769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199fdb-30cd-48e7-867b-477bacc703d3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:15.036776 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005083db-fbbf-4435-8650-0e211d578aa2 admin admin] RBAC: Authorizing `identity:get_group(group_id=80584f1064a74e0a8b82e2dd91025db0)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:15.037852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-005083db-fbbf-4435-8650-0e211d578aa2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:29:15.037852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Oct 24 13:29:15.037852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: Oct 24 13:29:15.039244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005083db-fbbf-4435-8650-0e211d578aa2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:15.041487 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 129/263] 10.222.0.20 () {56 vars in 1201 bytes} [Mon Oct 24 13:29:14 2022] GET /identity/v3/groups/80584f1064a74e0a8b82e2dd91025db0 => generated 226 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:15.043251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199fdb-30cd-48e7-867b-477bacc703d3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:15.046730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005083db-fbbf-4435-8650-0e211d578aa2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:15.048636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a659ec-955d-4729-8ffa-ebd188b7a714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:15.049559 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a659ec-955d-4729-8ffa-ebd188b7a714 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.050039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a659ec-955d-4729-8ffa-ebd188b7a714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.050517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a659ec-955d-4729-8ffa-ebd188b7a714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.051452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a659ec-955d-4729-8ffa-ebd188b7a714 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:15.051956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a659ec-955d-4729-8ffa-ebd188b7a714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:15.054568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-f2a659ec-955d-4729-8ffa-ebd188b7a714 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:29:15.054568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Oct 24 13:29:15.054568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: Oct 24 13:29:15.056115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a659ec-955d-4729-8ffa-ebd188b7a714 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:15.064642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005083db-fbbf-4435-8650-0e211d578aa2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:15.070904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 135/264] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Oct 24 13:29:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:15.076947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a659ec-955d-4729-8ffa-ebd188b7a714 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:15.077818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cdc09-744b-4e18-8012-9f4bc8aacdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:15.078502 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cdc09-744b-4e18-8012-9f4bc8aacdce admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.078784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cdc09-744b-4e18-8012-9f4bc8aacdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.079056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cdc09-744b-4e18-8012-9f4bc8aacdce admin admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.082872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cdc09-744b-4e18-8012-9f4bc8aacdce admin admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:15.086334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cdc09-744b-4e18-8012-9f4bc8aacdce admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:15.087119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 130/265] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Oct 24 13:29:15 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:15.090419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a659ec-955d-4729-8ffa-ebd188b7a714 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:15.092563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cdc09-744b-4e18-8012-9f4bc8aacdce admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:15.095760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430a0a5-211c-4544-9fab-79566753c2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:15.096629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430a0a5-211c-4544-9fab-79566753c2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.097220 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430a0a5-211c-4544-9fab-79566753c2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.097695 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430a0a5-211c-4544-9fab-79566753c2b7 admin admin] PATH_INFO: `/v3/groups/80584f1064a74e0a8b82e2dd91025db0` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.099196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430a0a5-211c-4544-9fab-79566753c2b7 admin admin] RBAC: Authorizing `identity:get_group(group_id=80584f1064a74e0a8b82e2dd91025db0)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:15.100719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-3430a0a5-211c-4544-9fab-79566753c2b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:29:15.100719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Oct 24 13:29:15.100719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: Oct 24 13:29:15.102917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:15.105785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430a0a5-211c-4544-9fab-79566753c2b7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:15.105934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 136/266] 10.222.0.20 () {56 vars in 1201 bytes} [Mon Oct 24 13:29:15 2022] GET /identity/v3/groups/80584f1064a74e0a8b82e2dd91025db0 => generated 226 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:15.114632 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430a0a5-211c-4544-9fab-79566753c2b7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:15.115855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cdc09-744b-4e18-8012-9f4bc8aacdce admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:15.127430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430a0a5-211c-4544-9fab-79566753c2b7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:15.132823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b422d3-88b9-40a2-93ba-42cde463a6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:15.133451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b422d3-88b9-40a2-93ba-42cde463a6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.133729 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b422d3-88b9-40a2-93ba-42cde463a6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.134091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b422d3-88b9-40a2-93ba-42cde463a6b0 admin admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.134384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae676d6-ca95-4e8c-a576-50a40fc56d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:15.135267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b422d3-88b9-40a2-93ba-42cde463a6b0 admin admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:15.135486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae676d6-ca95-4e8c-a576-50a40fc56d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.135955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae676d6-ca95-4e8c-a576-50a40fc56d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.136417 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae676d6-ca95-4e8c-a576-50a40fc56d25 admin admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790/groups/80584f1064a74e0a8b82e2dd91025db0/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.137646 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b422d3-88b9-40a2-93ba-42cde463a6b0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:15.138845 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae676d6-ca95-4e8c-a576-50a40fc56d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a634485bda4247f4996f560aa4394790, group_id=80584f1064a74e0a8b82e2dd91025db0, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:15.139704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 137/267] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Oct 24 13:29:15 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:15.140607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-dae676d6-ca95-4e8c-a576-50a40fc56d25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:29:15.140607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Oct 24 13:29:15.140607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: Oct 24 13:29:15.143008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae676d6-ca95-4e8c-a576-50a40fc56d25 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:15.145207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b422d3-88b9-40a2-93ba-42cde463a6b0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:15.158033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b422d3-88b9-40a2-93ba-42cde463a6b0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:15.166848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8778230-ddcd-4f4d-a342-fadd42874841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:15.167711 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8778230-ddcd-4f4d-a342-fadd42874841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.167910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8778230-ddcd-4f4d-a342-fadd42874841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.168069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8778230-ddcd-4f4d-a342-fadd42874841 admin admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790/groups/80584f1064a74e0a8b82e2dd91025db0/roles/f948e6f6ee4f42de92195aee35b05641` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.179061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8778230-ddcd-4f4d-a342-fadd42874841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a634485bda4247f4996f560aa4394790, group_id=80584f1064a74e0a8b82e2dd91025db0, role_id=f948e6f6ee4f42de92195aee35b05641)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:15.180104 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-d8778230-ddcd-4f4d-a342-fadd42874841 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:29:15.180104 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Oct 24 13:29:15.180104 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: Oct 24 13:29:15.180628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 131/268] 10.222.0.20 () {58 vars in 1463 bytes} [Mon Oct 24 13:29:15 2022] PUT /identity/v3/projects/a634485bda4247f4996f560aa4394790/groups/80584f1064a74e0a8b82e2dd91025db0/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:15.182174 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8778230-ddcd-4f4d-a342-fadd42874841 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:15.221256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 138/269] 10.222.0.20 () {58 vars in 1463 bytes} [Mon Oct 24 13:29:15 2022] PUT /identity/v3/projects/a634485bda4247f4996f560aa4394790/groups/80584f1064a74e0a8b82e2dd91025db0/roles/f948e6f6ee4f42de92195aee35b05641 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:29:15.502591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a641ad-dfc9-43f3-8229-0f9340ef926d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.503098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a641ad-dfc9-43f3-8229-0f9340ef926d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.503098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a641ad-dfc9-43f3-8229-0f9340ef926d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.504596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 132/270] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:15.510989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627619-64ce-49f9-9203-8bac2406c62e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.511922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627619-64ce-49f9-9203-8bac2406c62e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.512426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627619-64ce-49f9-9203-8bac2406c62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.538076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ee627619-64ce-49f9-9203-8bac2406c62e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:15.539641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ee627619-64ce-49f9-9203-8bac2406c62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:15.612513 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 139/271] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 13:29:15.729768 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5229c-2b3f-4f70-9ea8-71b4622b6fe8 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.730119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5229c-2b3f-4f70-9ea8-71b4622b6fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.730356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5229c-2b3f-4f70-9ea8-71b4622b6fe8 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.731958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 133/272] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:15.738218 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4685e8c-56c9-4433-947a-1247383275a4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:15.738757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4685e8c-56c9-4433-947a-1247383275a4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:15.739247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4685e8c-56c9-4433-947a-1247383275a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:15.762461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e4685e8c-56c9-4433-947a-1247383275a4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:15.763924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e4685e8c-56c9-4433-947a-1247383275a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:15.800859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:15.855984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 140/273] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:16.002023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5229c-2b3f-4f70-9ea8-71b4622b6fe8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:16.015426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5229c-2b3f-4f70-9ea8-71b4622b6fe8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:16.021318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd730f5-b85d-43bd-b192-c2a2c0de44e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:16.022127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd730f5-b85d-43bd-b192-c2a2c0de44e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:16.022539 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd730f5-b85d-43bd-b192-c2a2c0de44e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:16.022817 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd730f5-b85d-43bd-b192-c2a2c0de44e6 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:16.037801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd730f5-b85d-43bd-b192-c2a2c0de44e6 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-fcd730f5-b85d-43bd-b192-c2a2c0de44e6 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:16.042941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:16.043787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:16.044563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 134/274] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Oct 24 13:29:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:16.051955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4685e8c-56c9-4433-947a-1247383275a4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:16.064342 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4685e8c-56c9-4433-947a-1247383275a4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:16.069662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ab033-9064-4ade-8d2b-f7dd05e46843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:16.070537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ab033-9064-4ade-8d2b-f7dd05e46843 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:16.071011 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ab033-9064-4ade-8d2b-f7dd05e46843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:16.071467 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ab033-9064-4ade-8d2b-f7dd05e46843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:16.072336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ab033-9064-4ade-8d2b-f7dd05e46843 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:16.072899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ab033-9064-4ade-8d2b-f7dd05e46843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:16.077302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ab033-9064-4ade-8d2b-f7dd05e46843 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:16.092617 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 141/275] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:29:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:16.098570 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd730f5-b85d-43bd-b192-c2a2c0de44e6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:16.115939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd730f5-b85d-43bd-b192-c2a2c0de44e6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:16.124109 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922dfe6-bd01-464f-b8e2-5079792a6adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:16.124756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922dfe6-bd01-464f-b8e2-5079792a6adc admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:16.125038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922dfe6-bd01-464f-b8e2-5079792a6adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:16.125313 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922dfe6-bd01-464f-b8e2-5079792a6adc admin admin] PATH_INFO: `/v3/groups/80584f1064a74e0a8b82e2dd91025db0` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:16.126492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922dfe6-bd01-464f-b8e2-5079792a6adc admin admin] RBAC: Authorizing `identity:get_group(group_id=80584f1064a74e0a8b82e2dd91025db0)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:16.128442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922dfe6-bd01-464f-b8e2-5079792a6adc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:16.130012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 135/276] 10.222.0.20 () {56 vars in 1201 bytes} [Mon Oct 24 13:29:16 2022] GET /identity/v3/groups/80584f1064a74e0a8b82e2dd91025db0 => generated 226 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:16.136027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ab033-9064-4ade-8d2b-f7dd05e46843 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:16.146616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ab033-9064-4ade-8d2b-f7dd05e46843 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:16.155074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d23b0b-9858-448f-a7a3-33eca2a7e717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:16.155969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d23b0b-9858-448f-a7a3-33eca2a7e717 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:16.156457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d23b0b-9858-448f-a7a3-33eca2a7e717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:16.156984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d23b0b-9858-448f-a7a3-33eca2a7e717 admin admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:16.158373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d23b0b-9858-448f-a7a3-33eca2a7e717 admin admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:16.160717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d23b0b-9858-448f-a7a3-33eca2a7e717 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:16.163269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 142/277] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Oct 24 13:29:16 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:16.169468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922dfe6-bd01-464f-b8e2-5079792a6adc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:16.187947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922dfe6-bd01-464f-b8e2-5079792a6adc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:16.208299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9624e6-745c-4c30-b38b-c8f7c074d9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:16.209081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9624e6-745c-4c30-b38b-c8f7c074d9c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:16.209368 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9624e6-745c-4c30-b38b-c8f7c074d9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:16.209639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9624e6-745c-4c30-b38b-c8f7c074d9c3 admin admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555/groups/80584f1064a74e0a8b82e2dd91025db0/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:16.215494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9624e6-745c-4c30-b38b-c8f7c074d9c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5203564be545e1a1d8a33a58625555, group_id=80584f1064a74e0a8b82e2dd91025db0, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:16.218408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9624e6-745c-4c30-b38b-c8f7c074d9c3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:16.247876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 136/278] 10.222.0.20 () {58 vars in 1463 bytes} [Mon Oct 24 13:29:16 2022] PUT /identity/v3/projects/7d5203564be545e1a1d8a33a58625555/groups/80584f1064a74e0a8b82e2dd91025db0/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:16.405486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d23b0b-9858-448f-a7a3-33eca2a7e717 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:16.452376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc756-ea09-450f-aa5a-1a5a34734fa6 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:16.452731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc756-ea09-450f-aa5a-1a5a34734fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:16.452935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc756-ea09-450f-aa5a-1a5a34734fa6 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:16.454544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 137/279] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:16.471762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7efed0-2f44-431a-8010-59064d25bf28 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:16.472070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7efed0-2f44-431a-8010-59064d25bf28 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:16.472306 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7efed0-2f44-431a-8010-59064d25bf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:16.515925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1e7efed0-2f44-431a-8010-59064d25bf28 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:16.517349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1e7efed0-2f44-431a-8010-59064d25bf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:16.571949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d23b0b-9858-448f-a7a3-33eca2a7e717 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:16.608704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 138/280] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 13:29:16.629845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2452e4-ea7a-4c9e-80b8-8db4e08a328b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:16.631621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2452e4-ea7a-4c9e-80b8-8db4e08a328b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:16.632194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2452e4-ea7a-4c9e-80b8-8db4e08a328b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:16.632659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2452e4-ea7a-4c9e-80b8-8db4e08a328b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:16.646604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2452e4-ea7a-4c9e-80b8-8db4e08a328b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-ff2452e4-ea7a-4c9e-80b8-8db4e08a328b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:16.652266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:16.653181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:29:16.653996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:16.656662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 143/281] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:16.663540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7efed0-2f44-431a-8010-59064d25bf28 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:16.777234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7efed0-2f44-431a-8010-59064d25bf28 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:16.788006 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:16.821631 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fccb454-8681-469b-b354-a6a96c627f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:16.822703 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccb454-8681-469b-b354-a6a96c627f05 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:16.823176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccb454-8681-469b-b354-a6a96c627f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:16.823658 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fccb454-8681-469b-b354-a6a96c627f05 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:16.827512 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fccb454-8681-469b-b354-a6a96c627f05 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:16.828031 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fccb454-8681-469b-b354-a6a96c627f05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:16.829734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fccb454-8681-469b-b354-a6a96c627f05 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:16.840240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 139/282] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:29:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:16.846241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2452e4-ea7a-4c9e-80b8-8db4e08a328b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:16.869142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:16.995233 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2452e4-ea7a-4c9e-80b8-8db4e08a328b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:17.042508 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3739c1-1f69-4fe1-b5be-30c158901ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:17.043157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3739c1-1f69-4fe1-b5be-30c158901ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.043438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3739c1-1f69-4fe1-b5be-30c158901ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.043713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3739c1-1f69-4fe1-b5be-30c158901ad9 None admin] PATH_INFO: `/v3/users/d3c0844c4a224dcca9e1aafec42f249e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:17.044871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3739c1-1f69-4fe1-b5be-30c158901ad9 None admin] RBAC: Authorizing `identity:get_user(user_id=d3c0844c4a224dcca9e1aafec42f249e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:17.046344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3739c1-1f69-4fe1-b5be-30c158901ad9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:17.048406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 144/283] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:29:16 2022] GET /identity/v3/users/d3c0844c4a224dcca9e1aafec42f249e => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:17.055349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fccb454-8681-469b-b354-a6a96c627f05 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:17.149066 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3739c1-1f69-4fe1-b5be-30c158901ad9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:17.167146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3739c1-1f69-4fe1-b5be-30c158901ad9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:17.175682 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96b8a0-0fa4-4e6a-8734-901a667b5e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:17.176322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96b8a0-0fa4-4e6a-8734-901a667b5e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.176605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96b8a0-0fa4-4e6a-8734-901a667b5e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.176878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96b8a0-0fa4-4e6a-8734-901a667b5e34 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:17.213928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96b8a0-0fa4-4e6a-8734-901a667b5e34 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-2a96b8a0-0fa4-4e6a-8734-901a667b5e34 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:17.217033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:17.217891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:29:17.218697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:17.223613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 145/284] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Oct 24 13:29:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 75 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:17.225822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fccb454-8681-469b-b354-a6a96c627f05 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:17.231468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96b8a0-0fa4-4e6a-8734-901a667b5e34 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:17.246009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96b8a0-0fa4-4e6a-8734-901a667b5e34 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:17.259707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d5851-ed82-4dfa-a8f5-064156a7e6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:17.260333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d5851-ed82-4dfa-a8f5-064156a7e6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.260617 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d5851-ed82-4dfa-a8f5-064156a7e6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.260887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d5851-ed82-4dfa-a8f5-064156a7e6a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:17.262193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d5851-ed82-4dfa-a8f5-064156a7e6a7 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:17.262193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d5851-ed82-4dfa-a8f5-064156a7e6a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:17.263498 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d5851-ed82-4dfa-a8f5-064156a7e6a7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:17.273710 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 146/285] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Oct 24 13:29:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:17.276039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6852e-6da1-4900-bd21-0894b8befccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:17.276919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6852e-6da1-4900-bd21-0894b8befccb None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.277387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6852e-6da1-4900-bd21-0894b8befccb None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.277971 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6852e-6da1-4900-bd21-0894b8befccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:17.278870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d5851-ed82-4dfa-a8f5-064156a7e6a7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:17.279104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6852e-6da1-4900-bd21-0894b8befccb None admin] RBAC: Adding query filter params (role.id=84fba578b2cd41e6958097aec20a6f6e, user.id=d3c0844c4a224dcca9e1aafec42f249e, scope.system=all) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:17.279739 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6852e-6da1-4900-bd21-0894b8befccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:17.281408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6852e-6da1-4900-bd21-0894b8befccb None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:17.297650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d5851-ed82-4dfa-a8f5-064156a7e6a7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:17.306727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 140/286] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Oct 24 13:29:17 2022] GET /identity/v3/role_assignments?user.id=d3c0844c4a224dcca9e1aafec42f249e&scope.system=all&role.id=84fba578b2cd41e6958097aec20a6f6e => generated 510 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:17.308447 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46053e25-0e79-4d95-ab24-c23b268839ff None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.308934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46053e25-0e79-4d95-ab24-c23b268839ff None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.309395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46053e25-0e79-4d95-ab24-c23b268839ff None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:17.310923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 141/287] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:17.316442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c37cf-76ee-4393-acab-a341ba1cc9bd None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.317042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c37cf-76ee-4393-acab-a341ba1cc9bd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.317527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c37cf-76ee-4393-acab-a341ba1cc9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:17.319077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979b15c-1ab3-4ef1-a652-3c11a45d3b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:17.320333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979b15c-1ab3-4ef1-a652-3c11a45d3b40 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.320807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979b15c-1ab3-4ef1-a652-3c11a45d3b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.321274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979b15c-1ab3-4ef1-a652-3c11a45d3b40 admin admin] PATH_INFO: `/v3/groups/80584f1064a74e0a8b82e2dd91025db0` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:17.323064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979b15c-1ab3-4ef1-a652-3c11a45d3b40 admin admin] RBAC: Authorizing `identity:get_group(group_id=80584f1064a74e0a8b82e2dd91025db0)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:17.330200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979b15c-1ab3-4ef1-a652-3c11a45d3b40 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:17.330200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 147/288] 10.222.0.20 () {56 vars in 1201 bytes} [Mon Oct 24 13:29:17 2022] GET /identity/v3/groups/80584f1064a74e0a8b82e2dd91025db0 => generated 226 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:17.334615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979b15c-1ab3-4ef1-a652-3c11a45d3b40 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:17.346348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-661c37cf-76ee-4393-acab-a341ba1cc9bd None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:17.347992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-661c37cf-76ee-4393-acab-a341ba1cc9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:17.353131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979b15c-1ab3-4ef1-a652-3c11a45d3b40 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:17.364805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 142/289] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 13:29:17.371014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbf44f-2d74-42ee-b46d-fe63e64d2a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:17.371639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbf44f-2d74-42ee-b46d-fe63e64d2a3b admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.371921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbf44f-2d74-42ee-b46d-fe63e64d2a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.372194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbf44f-2d74-42ee-b46d-fe63e64d2a3b admin admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:17.373427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbf44f-2d74-42ee-b46d-fe63e64d2a3b admin admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:17.375534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbf44f-2d74-42ee-b46d-fe63e64d2a3b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:17.377205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 148/290] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Oct 24 13:29:17 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:17.383152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c37cf-76ee-4393-acab-a341ba1cc9bd None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:17.397293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c37cf-76ee-4393-acab-a341ba1cc9bd None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:17.404987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9f9a0-876b-4cc0-be55-1e4cc8ce9b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:17.406013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9f9a0-876b-4cc0-be55-1e4cc8ce9b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.406506 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9f9a0-876b-4cc0-be55-1e4cc8ce9b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.407078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9f9a0-876b-4cc0-be55-1e4cc8ce9b04 admin admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555/groups/80584f1064a74e0a8b82e2dd91025db0/roles/f948e6f6ee4f42de92195aee35b05641` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:17.409572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9f9a0-876b-4cc0-be55-1e4cc8ce9b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5203564be545e1a1d8a33a58625555, group_id=80584f1064a74e0a8b82e2dd91025db0, role_id=f948e6f6ee4f42de92195aee35b05641)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:17.413758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9f9a0-876b-4cc0-be55-1e4cc8ce9b04 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:17.440212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 143/291] 10.222.0.20 () {58 vars in 1463 bytes} [Mon Oct 24 13:29:17 2022] PUT /identity/v3/projects/7d5203564be545e1a1d8a33a58625555/groups/80584f1064a74e0a8b82e2dd91025db0/roles/f948e6f6ee4f42de92195aee35b05641 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:17.805337 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbf44f-2d74-42ee-b46d-fe63e64d2a3b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:17.815953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbf44f-2d74-42ee-b46d-fe63e64d2a3b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:17.873911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:17.884105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fa26f-ebda-45bf-922f-19b33cc22207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:17.884800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fa26f-ebda-45bf-922f-19b33cc22207 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.885081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fa26f-ebda-45bf-922f-19b33cc22207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.885349 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fa26f-ebda-45bf-922f-19b33cc22207 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:17.902863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4fa26f-ebda-45bf-922f-19b33cc22207 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-dc4fa26f-ebda-45bf-922f-19b33cc22207 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:17.907727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:17.908581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:17.909376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 149/292] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Oct 24 13:29:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 104 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:17.915265 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9f9a0-876b-4cc0-be55-1e4cc8ce9b04 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:17.936557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9f9a0-876b-4cc0-be55-1e4cc8ce9b04 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:17.952265 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b21d58-99a1-4667-973b-0f2cf57b8f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:17.952891 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b21d58-99a1-4667-973b-0f2cf57b8f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.953172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b21d58-99a1-4667-973b-0f2cf57b8f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.953443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b21d58-99a1-4667-973b-0f2cf57b8f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:17.954120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b21d58-99a1-4667-973b-0f2cf57b8f17 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:17.954453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b21d58-99a1-4667-973b-0f2cf57b8f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:17.956050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b21d58-99a1-4667-973b-0f2cf57b8f17 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:17.966621 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:17.971253 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 144/293] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Oct 24 13:29:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:17.977196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fa26f-ebda-45bf-922f-19b33cc22207 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:17.992261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fa26f-ebda-45bf-922f-19b33cc22207 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:17.997832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5e109-6750-4fc8-8453-26ec2cad6092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:17.998560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5e109-6750-4fc8-8453-26ec2cad6092 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:17.998841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5e109-6750-4fc8-8453-26ec2cad6092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:17.999110 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5e109-6750-4fc8-8453-26ec2cad6092 admin admin] PATH_INFO: `/v3/groups/2846858e65d74896bf528feb5c6634a1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:18.007940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5e109-6750-4fc8-8453-26ec2cad6092 admin admin] RBAC: Authorizing `identity:get_group(group_id=2846858e65d74896bf528feb5c6634a1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:18.009938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5e109-6750-4fc8-8453-26ec2cad6092 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:18.011955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 150/294] 10.222.0.20 () {56 vars in 1201 bytes} [Mon Oct 24 13:29:17 2022] GET /identity/v3/groups/2846858e65d74896bf528feb5c6634a1 => generated 229 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:18.017186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b21d58-99a1-4667-973b-0f2cf57b8f17 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:18.031259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b21d58-99a1-4667-973b-0f2cf57b8f17 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:18.038309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c621d-1ab8-4fac-8934-e9c453fec9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:18.038943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c621d-1ab8-4fac-8934-e9c453fec9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:18.039225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c621d-1ab8-4fac-8934-e9c453fec9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:18.039498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c621d-1ab8-4fac-8934-e9c453fec9c7 admin admin] PATH_INFO: `/v3/projects/73bbcd87686646c2a2c2fdd8f0e7bdd8` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:18.045726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c621d-1ab8-4fac-8934-e9c453fec9c7 admin admin] RBAC: Authorizing `identity:get_project(project_id=73bbcd87686646c2a2c2fdd8f0e7bdd8)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:18.047657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c621d-1ab8-4fac-8934-e9c453fec9c7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:18.049637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 145/295] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Oct 24 13:29:18 2022] GET /identity/v3/projects/73bbcd87686646c2a2c2fdd8f0e7bdd8 => generated 344 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:18.055937 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5e109-6750-4fc8-8453-26ec2cad6092 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:18.067397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5e109-6750-4fc8-8453-26ec2cad6092 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:18.075688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e4e3a-cce2-43ed-b514-18229a41cc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:18.076360 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e4e3a-cce2-43ed-b514-18229a41cc96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:18.076639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e4e3a-cce2-43ed-b514-18229a41cc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:18.076909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e4e3a-cce2-43ed-b514-18229a41cc96 admin admin] PATH_INFO: `/v3/projects/73bbcd87686646c2a2c2fdd8f0e7bdd8/groups/2846858e65d74896bf528feb5c6634a1/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:18.078884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e4e3a-cce2-43ed-b514-18229a41cc96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73bbcd87686646c2a2c2fdd8f0e7bdd8, group_id=2846858e65d74896bf528feb5c6634a1, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:18.081556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e4e3a-cce2-43ed-b514-18229a41cc96 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:18.106301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 151/296] 10.222.0.20 () {58 vars in 1463 bytes} [Mon Oct 24 13:29:18 2022] PUT /identity/v3/projects/73bbcd87686646c2a2c2fdd8f0e7bdd8/groups/2846858e65d74896bf528feb5c6634a1/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:18.455320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682d6e2-5ce4-4e13-a6b6-63fe74c032fe None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:18.455652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682d6e2-5ce4-4e13-a6b6-63fe74c032fe None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:18.455840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682d6e2-5ce4-4e13-a6b6-63fe74c032fe None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:18.457437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 146/297] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:18.463312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf34ece-6a21-4b94-aea8-b0822c9c7a59 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:18.463601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf34ece-6a21-4b94-aea8-b0822c9c7a59 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:18.463963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf34ece-6a21-4b94-aea8-b0822c9c7a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:18.490978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7bf34ece-6a21-4b94-aea8-b0822c9c7a59 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:18.492192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7bf34ece-6a21-4b94-aea8-b0822c9c7a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:18.587816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 152/298] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:18.862025 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1acec4-1af1-4d3a-9b77-edcc03abf346 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:18.862370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1acec4-1af1-4d3a-9b77-edcc03abf346 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:18.862607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1acec4-1af1-4d3a-9b77-edcc03abf346 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:18.864195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 147/299] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:18.869961 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd0e74-c7c2-4c9d-bd8f-b76de349ce02 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:18.870443 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd0e74-c7c2-4c9d-bd8f-b76de349ce02 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:18.870620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd0e74-c7c2-4c9d-bd8f-b76de349ce02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:18.894948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-63fd0e74-c7c2-4c9d-bd8f-b76de349ce02 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:18.896128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-63fd0e74-c7c2-4c9d-bd8f-b76de349ce02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:18.985264 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 153/300] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:29:19.074691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1acec4-1af1-4d3a-9b77-edcc03abf346 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:19.230038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1acec4-1af1-4d3a-9b77-edcc03abf346 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:19.276744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621eb22-83a2-4555-ab62-3b989ec9d21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:19.277382 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621eb22-83a2-4555-ab62-3b989ec9d21a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:19.277663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621eb22-83a2-4555-ab62-3b989ec9d21a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:19.277934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621eb22-83a2-4555-ab62-3b989ec9d21a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:19.295494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621eb22-83a2-4555-ab62-3b989ec9d21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-d621eb22-83a2-4555-ab62-3b989ec9d21a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:19.300370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:19.301217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:19.302043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 148/301] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:29:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:19.309637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd0e74-c7c2-4c9d-bd8f-b76de349ce02 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:19.347614 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621eb22-83a2-4555-ab62-3b989ec9d21a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:19.354070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:19.470797 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:19.488395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd0e74-c7c2-4c9d-bd8f-b76de349ce02 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:19.538414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621eb22-83a2-4555-ab62-3b989ec9d21a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:19.546260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf64a7b-111f-4696-b129-69da71693c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:19.546884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64a7b-111f-4696-b129-69da71693c55 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:19.547161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64a7b-111f-4696-b129-69da71693c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:19.547430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64a7b-111f-4696-b129-69da71693c55 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:19.548163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf64a7b-111f-4696-b129-69da71693c55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:19.548467 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf64a7b-111f-4696-b129-69da71693c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:19.550796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf64a7b-111f-4696-b129-69da71693c55 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:19.558852 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 154/302] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:19.561113 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c30b97-7cb2-4167-8c98-1cffc2f404e6 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:19.561400 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c30b97-7cb2-4167-8c98-1cffc2f404e6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:19.561668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c30b97-7cb2-4167-8c98-1cffc2f404e6 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:19.567201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 155/303] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:19.568067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c30b97-7cb2-4167-8c98-1cffc2f404e6 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:19.584933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee91a22-184e-43b6-84fa-97ac753fc0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:19.586146 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee91a22-184e-43b6-84fa-97ac753fc0dd None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:19.586659 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee91a22-184e-43b6-84fa-97ac753fc0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:19.587392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee91a22-184e-43b6-84fa-97ac753fc0dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:19.602560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee91a22-184e-43b6-84fa-97ac753fc0dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-7ee91a22-184e-43b6-84fa-97ac753fc0dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:19.609510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:19.610441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:19.611242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:19.616730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 149/304] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:19.618383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becd908-5cfc-43a8-8fff-bb38cf35b08d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:19.618819 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becd908-5cfc-43a8-8fff-bb38cf35b08d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:19.619837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becd908-5cfc-43a8-8fff-bb38cf35b08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:19.644140 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4becd908-5cfc-43a8-8fff-bb38cf35b08d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:19.645257 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4becd908-5cfc-43a8-8fff-bb38cf35b08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:19.679020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:19.694481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c30b97-7cb2-4167-8c98-1cffc2f404e6 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:19.729470 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 150/305] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:19.730880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becd908-5cfc-43a8-8fff-bb38cf35b08d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:19.731671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc727357-eef7-46c3-9c34-97bd74b96526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:19.732293 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727357-eef7-46c3-9c34-97bd74b96526 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:19.732575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727357-eef7-46c3-9c34-97bd74b96526 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:19.732849 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc727357-eef7-46c3-9c34-97bd74b96526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:19.788941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:19.823381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc727357-eef7-46c3-9c34-97bd74b96526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:19.825609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc727357-eef7-46c3-9c34-97bd74b96526 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:19.879591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becd908-5cfc-43a8-8fff-bb38cf35b08d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:19.951503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad34143-c323-4df9-a00f-ba30e2a1d64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:19.952130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad34143-c323-4df9-a00f-ba30e2a1d64a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:19.952409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad34143-c323-4df9-a00f-ba30e2a1d64a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:19.952678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad34143-c323-4df9-a00f-ba30e2a1d64a None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:19.953354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad34143-c323-4df9-a00f-ba30e2a1d64a None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:19.953654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad34143-c323-4df9-a00f-ba30e2a1d64a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:19.955059 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad34143-c323-4df9-a00f-ba30e2a1d64a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:19.969831 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 151/306] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:29:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:19.975519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad34143-c323-4df9-a00f-ba30e2a1d64a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:19.978750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:20.002043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 156/307] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:29:20.008112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc727357-eef7-46c3-9c34-97bd74b96526 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:20.136233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad34143-c323-4df9-a00f-ba30e2a1d64a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:20.146039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:20.164169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc727357-eef7-46c3-9c34-97bd74b96526 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:20.177210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df8a98-3b8c-4104-8e47-87d452f5c52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:20.178964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df8a98-3b8c-4104-8e47-87d452f5c52f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:20.179247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df8a98-3b8c-4104-8e47-87d452f5c52f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:20.179514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df8a98-3b8c-4104-8e47-87d452f5c52f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:20.215906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc14ba-c206-40e6-a663-153d8436583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:20.216924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc14ba-c206-40e6-a663-153d8436583b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:20.217274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc14ba-c206-40e6-a663-153d8436583b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:20.217939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc14ba-c206-40e6-a663-153d8436583b None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:20.219699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc14ba-c206-40e6-a663-153d8436583b None admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:20.221639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc14ba-c206-40e6-a663-153d8436583b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:20.225209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 157/308] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:29:20 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:20.226414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc14ba-c206-40e6-a663-153d8436583b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:20.280281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df8a98-3b8c-4104-8e47-87d452f5c52f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:20.281943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df8a98-3b8c-4104-8e47-87d452f5c52f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:20.310570 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:20.379971 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc14ba-c206-40e6-a663-153d8436583b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:20.411008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 152/309] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 24 13:29:20.412607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df8a98-3b8c-4104-8e47-87d452f5c52f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:20.421972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9628cc6-4241-4a89-bb07-6e78ef54d85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:20.422765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9628cc6-4241-4a89-bb07-6e78ef54d85c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:20.422998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9628cc6-4241-4a89-bb07-6e78ef54d85c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:20.423265 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9628cc6-4241-4a89-bb07-6e78ef54d85c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:20.423757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:20.436404 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9628cc6-4241-4a89-bb07-6e78ef54d85c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-a9628cc6-4241-4a89-bb07-6e78ef54d85c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:20.441652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:20.442814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 158/310] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:20.443807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9628cc6-4241-4a89-bb07-6e78ef54d85c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:20.534705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df8a98-3b8c-4104-8e47-87d452f5c52f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:20.561720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9628cc6-4241-4a89-bb07-6e78ef54d85c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:20.577576 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7883666-c443-45c9-a23a-6077fdcb4d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:20.579070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7883666-c443-45c9-a23a-6077fdcb4d04 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:20.579544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7883666-c443-45c9-a23a-6077fdcb4d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:20.580020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7883666-c443-45c9-a23a-6077fdcb4d04 None admin] PATH_INFO: `/v3/groups/80584f1064a74e0a8b82e2dd91025db0` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:20.581769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7883666-c443-45c9-a23a-6077fdcb4d04 None admin] RBAC: Authorizing `identity:get_group(group_id=80584f1064a74e0a8b82e2dd91025db0)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:20.582252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:20.585379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7883666-c443-45c9-a23a-6077fdcb4d04 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:20.587484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 153/311] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Oct 24 13:29:20 2022] GET /identity/v3/groups/80584f1064a74e0a8b82e2dd91025db0 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:20.588437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7883666-c443-45c9-a23a-6077fdcb4d04 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:20.603420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df070e-7680-42d6-a654-39b4b697bcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:20.604030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df070e-7680-42d6-a654-39b4b697bcbf None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:20.604304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df070e-7680-42d6-a654-39b4b697bcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:20.604569 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df070e-7680-42d6-a654-39b4b697bcbf None admin] PATH_INFO: `/v3/projects/a634485bda4247f4996f560aa4394790` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:20.606151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df070e-7680-42d6-a654-39b4b697bcbf None admin] RBAC: Authorizing `identity:get_project(project_id=a634485bda4247f4996f560aa4394790)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:20.607541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df070e-7680-42d6-a654-39b4b697bcbf None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:20.609192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 159/312] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:29:20 2022] GET /identity/v3/projects/a634485bda4247f4996f560aa4394790 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:20.610286 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112b028-5546-4826-86f7-dd4a7e4c1dda None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:20.610610 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112b028-5546-4826-86f7-dd4a7e4c1dda None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:20.610824 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112b028-5546-4826-86f7-dd4a7e4c1dda None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:20.612497 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 160/313] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:20.612944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112b028-5546-4826-86f7-dd4a7e4c1dda None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:20.689603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:20.719356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7883666-c443-45c9-a23a-6077fdcb4d04 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:20.749136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112b028-5546-4826-86f7-dd4a7e4c1dda None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:20.765869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b5472-bc27-405d-ba34-8ea56b2748ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:20.765869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b5472-bc27-405d-ba34-8ea56b2748ae None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:20.766186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b5472-bc27-405d-ba34-8ea56b2748ae None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:20.766265 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b5472-bc27-405d-ba34-8ea56b2748ae None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:20.766925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b5472-bc27-405d-ba34-8ea56b2748ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:20.767284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b5472-bc27-405d-ba34-8ea56b2748ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:20.769573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b5472-bc27-405d-ba34-8ea56b2748ae None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:20.779544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 154/314] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:29:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:20.780612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b34af-a2b8-48d5-a970-65ae1ee798d9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:20.780902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b34af-a2b8-48d5-a970-65ae1ee798d9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:20.781168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b34af-a2b8-48d5-a970-65ae1ee798d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:20.793489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72450467-6a43-4105-9560-c66b83d16390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:20.794024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72450467-6a43-4105-9560-c66b83d16390 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:20.794341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72450467-6a43-4105-9560-c66b83d16390 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:20.794612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72450467-6a43-4105-9560-c66b83d16390 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:20.795338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72450467-6a43-4105-9560-c66b83d16390 None admin] RBAC: Adding query filter params (group.id=80584f1064a74e0a8b82e2dd91025db0, role.id=f948e6f6ee4f42de92195aee35b05641, scope.project.id=a634485bda4247f4996f560aa4394790) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:20.795645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72450467-6a43-4105-9560-c66b83d16390 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:20.797058 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72450467-6a43-4105-9560-c66b83d16390 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:20.803943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 161/315] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Oct 24 13:29:20 2022] GET /identity/v3/role_assignments?group.id=80584f1064a74e0a8b82e2dd91025db0&scope.project.id=a634485bda4247f4996f560aa4394790&role.id=f948e6f6ee4f42de92195aee35b05641 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:20.804630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3c0b34af-a2b8-48d5-a970-65ae1ee798d9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:20.805937 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3c0b34af-a2b8-48d5-a970-65ae1ee798d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:20.806683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72450467-6a43-4105-9560-c66b83d16390 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:20.884620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:20.895589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 155/316] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:20.897652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b34af-a2b8-48d5-a970-65ae1ee798d9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:20.963830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72450467-6a43-4105-9560-c66b83d16390 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:20.981559 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:21.013245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d36d48-96a0-426f-8647-7395c72d3dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:21.014842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d36d48-96a0-426f-8647-7395c72d3dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:21.015298 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d36d48-96a0-426f-8647-7395c72d3dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:21.016010 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d36d48-96a0-426f-8647-7395c72d3dc5 None admin] PATH_INFO: `/v3/groups/80584f1064a74e0a8b82e2dd91025db0` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:21.018123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d36d48-96a0-426f-8647-7395c72d3dc5 None admin] RBAC: Authorizing `identity:get_group(group_id=80584f1064a74e0a8b82e2dd91025db0)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:21.020574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d36d48-96a0-426f-8647-7395c72d3dc5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:21.023237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 162/317] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Oct 24 13:29:20 2022] GET /identity/v3/groups/80584f1064a74e0a8b82e2dd91025db0 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:21.029624 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b34af-a2b8-48d5-a970-65ae1ee798d9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:21.029797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d36d48-96a0-426f-8647-7395c72d3dc5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:21.094250 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9572312-a7fd-4944-9d8f-345c8fd013e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:21.095244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9572312-a7fd-4944-9d8f-345c8fd013e4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:21.095749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9572312-a7fd-4944-9d8f-345c8fd013e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:21.096222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9572312-a7fd-4944-9d8f-345c8fd013e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:21.186755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9572312-a7fd-4944-9d8f-345c8fd013e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:21.188408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d36d48-96a0-426f-8647-7395c72d3dc5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:21.189212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9572312-a7fd-4944-9d8f-345c8fd013e4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:21.208501 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:21.242503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d18dc-8cc6-4ad4-93d3-5c5c7957862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:21.244308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d18dc-8cc6-4ad4-93d3-5c5c7957862b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:21.244308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d18dc-8cc6-4ad4-93d3-5c5c7957862b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:21.244308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d18dc-8cc6-4ad4-93d3-5c5c7957862b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:21.244548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d18dc-8cc6-4ad4-93d3-5c5c7957862b None admin] RBAC: Adding query filter params (group.id=80584f1064a74e0a8b82e2dd91025db0, role.id=09a932be0c1b481c8494705d2030fc2a, scope.project.id=a634485bda4247f4996f560aa4394790) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:21.244723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d18dc-8cc6-4ad4-93d3-5c5c7957862b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:21.246090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d18dc-8cc6-4ad4-93d3-5c5c7957862b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:21.254050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 163/318] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Oct 24 13:29:21 2022] GET /identity/v3/role_assignments?group.id=80584f1064a74e0a8b82e2dd91025db0&scope.project.id=a634485bda4247f4996f560aa4394790&role.id=09a932be0c1b481c8494705d2030fc2a => generated 611 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:21.255334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21181a39-6b46-4dcc-8aaf-3e759156e516 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:21.255721 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21181a39-6b46-4dcc-8aaf-3e759156e516 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:21.256231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21181a39-6b46-4dcc-8aaf-3e759156e516 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:21.257727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 164/319] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:21.263317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a027e1a-fe96-4080-b4c2-4bbf39f08bc1 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:21.263602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a027e1a-fe96-4080-b4c2-4bbf39f08bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:21.263882 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a027e1a-fe96-4080-b4c2-4bbf39f08bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:21.283717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0a027e1a-fe96-4080-b4c2-4bbf39f08bc1 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:21.284954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0a027e1a-fe96-4080-b4c2-4bbf39f08bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:21.286095 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:21.335771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 156/320] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:29:21.336334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9572312-a7fd-4944-9d8f-345c8fd013e4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:21.386149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 165/321] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:29:21.387018 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a027e1a-fe96-4080-b4c2-4bbf39f08bc1 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:21.491008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9572312-a7fd-4944-9d8f-345c8fd013e4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:21.515969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a027e1a-fe96-4080-b4c2-4bbf39f08bc1 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:21.530566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:21.537359 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a5af0-1bd1-4f8a-8b2b-072cffb28139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:21.539481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a5af0-1bd1-4f8a-8b2b-072cffb28139 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:21.539481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a5af0-1bd1-4f8a-8b2b-072cffb28139 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:21.539481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a5af0-1bd1-4f8a-8b2b-072cffb28139 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:21.556161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a5af0-1bd1-4f8a-8b2b-072cffb28139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-af1a5af0-1bd1-4f8a-8b2b-072cffb28139 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:21.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:21.561916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:21.562834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 157/322] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:29:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:21.567489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3197fd1-d125-42be-9196-d77c73b27063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:21.568709 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3197fd1-d125-42be-9196-d77c73b27063 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:21.569171 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3197fd1-d125-42be-9196-d77c73b27063 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:21.569750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3197fd1-d125-42be-9196-d77c73b27063 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:21.571489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3197fd1-d125-42be-9196-d77c73b27063 None admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:21.572959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3197fd1-d125-42be-9196-d77c73b27063 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:21.574889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 166/323] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:29:21 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:21.575689 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a5af0-1bd1-4f8a-8b2b-072cffb28139 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:21.582388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3197fd1-d125-42be-9196-d77c73b27063 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:21.614925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:21.707718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3197fd1-d125-42be-9196-d77c73b27063 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:21.711008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a5af0-1bd1-4f8a-8b2b-072cffb28139 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:21.750214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e76059-c612-4f84-92c3-9ee5172f39eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:21.750745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd0b68-e285-4d91-bcb2-d4dacd282d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:21.750862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e76059-c612-4f84-92c3-9ee5172f39eb None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:21.751174 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e76059-c612-4f84-92c3-9ee5172f39eb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:21.751289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd0b68-e285-4d91-bcb2-d4dacd282d96 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:21.751508 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e76059-c612-4f84-92c3-9ee5172f39eb None admin] PATH_INFO: `/v3/groups/80584f1064a74e0a8b82e2dd91025db0` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:21.751596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd0b68-e285-4d91-bcb2-d4dacd282d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:21.751780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd0b68-e285-4d91-bcb2-d4dacd282d96 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:21.752446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd0b68-e285-4d91-bcb2-d4dacd282d96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:21.752642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e76059-c612-4f84-92c3-9ee5172f39eb None admin] RBAC: Authorizing `identity:get_group(group_id=80584f1064a74e0a8b82e2dd91025db0)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:21.752730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd0b68-e285-4d91-bcb2-d4dacd282d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:21.754099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd0b68-e285-4d91-bcb2-d4dacd282d96 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:21.754275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e76059-c612-4f84-92c3-9ee5172f39eb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:21.756896 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 167/324] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Oct 24 13:29:21 2022] GET /identity/v3/groups/80584f1064a74e0a8b82e2dd91025db0 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:21.756896 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e76059-c612-4f84-92c3-9ee5172f39eb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:21.766390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 158/325] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:21.767163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd0b68-e285-4d91-bcb2-d4dacd282d96 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:21.821955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:21.869771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:21.882297 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd0b68-e285-4d91-bcb2-d4dacd282d96 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:21.894337 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e76059-c612-4f84-92c3-9ee5172f39eb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:21.925163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4b0e2-feb3-408d-99b2-7f895bd7500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:21.925779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4b0e2-feb3-408d-99b2-7f895bd7500a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:21.926056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4b0e2-feb3-408d-99b2-7f895bd7500a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:21.926362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4b0e2-feb3-408d-99b2-7f895bd7500a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:21.927084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4b0e2-feb3-408d-99b2-7f895bd7500a None admin] RBAC: Adding query filter params (group.id=80584f1064a74e0a8b82e2dd91025db0, role.id=09a932be0c1b481c8494705d2030fc2a, scope.project.id=7d5203564be545e1a1d8a33a58625555) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:21.927389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4b0e2-feb3-408d-99b2-7f895bd7500a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:21.929399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4b0e2-feb3-408d-99b2-7f895bd7500a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:21.936578 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 159/326] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Oct 24 13:29:21 2022] GET /identity/v3/role_assignments?group.id=80584f1064a74e0a8b82e2dd91025db0&scope.project.id=7d5203564be545e1a1d8a33a58625555&role.id=09a932be0c1b481c8494705d2030fc2a => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:21.937400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4b0e2-feb3-408d-99b2-7f895bd7500a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:21.938802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895119b-1823-4e29-92db-d215e7b7fff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:21.939470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895119b-1823-4e29-92db-d215e7b7fff6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:21.939788 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895119b-1823-4e29-92db-d215e7b7fff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:21.940098 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895119b-1823-4e29-92db-d215e7b7fff6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:21.959507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895119b-1823-4e29-92db-d215e7b7fff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-0895119b-1823-4e29-92db-d215e7b7fff6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:21.964627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:21.965586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:21.966593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 168/327] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:29:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:21.971994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895119b-1823-4e29-92db-d215e7b7fff6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:22.093878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4b0e2-feb3-408d-99b2-7f895bd7500a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:22.121478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895119b-1823-4e29-92db-d215e7b7fff6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:22.132585 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:22.145952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf7213-b02a-4bfc-bb7d-7d5ed67f8f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:22.147002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7213-b02a-4bfc-bb7d-7d5ed67f8f5a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:22.147288 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7213-b02a-4bfc-bb7d-7d5ed67f8f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:22.147565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7213-b02a-4bfc-bb7d-7d5ed67f8f5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:22.168957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6b071-0b2e-41c8-9dfc-888ce03d5c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:22.169575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6b071-0b2e-41c8-9dfc-888ce03d5c35 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:22.169851 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6b071-0b2e-41c8-9dfc-888ce03d5c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:22.170117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6b071-0b2e-41c8-9dfc-888ce03d5c35 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:22.170856 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6b071-0b2e-41c8-9dfc-888ce03d5c35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:22.171151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6b071-0b2e-41c8-9dfc-888ce03d5c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:22.172618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6b071-0b2e-41c8-9dfc-888ce03d5c35 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:22.174460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:22.183968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 169/328] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:22.189835 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6b071-0b2e-41c8-9dfc-888ce03d5c35 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:22.244465 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf7213-b02a-4bfc-bb7d-7d5ed67f8f5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:22.245925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf7213-b02a-4bfc-bb7d-7d5ed67f8f5a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:22.313916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6b071-0b2e-41c8-9dfc-888ce03d5c35 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:22.358109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55398d-7a70-46eb-9f6c-70117e3d6d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:22.359295 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55398d-7a70-46eb-9f6c-70117e3d6d43 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:22.359699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55398d-7a70-46eb-9f6c-70117e3d6d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:22.360092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55398d-7a70-46eb-9f6c-70117e3d6d43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:22.361288 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 160/329] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:29:22.367444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf7213-b02a-4bfc-bb7d-7d5ed67f8f5a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:22.399239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:22.438053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:22.439565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55398d-7a70-46eb-9f6c-70117e3d6d43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:22.441277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55398d-7a70-46eb-9f6c-70117e3d6d43 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:22.492395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf7213-b02a-4bfc-bb7d-7d5ed67f8f5a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:22.526824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beadf86-dabd-422a-99a7-216f4a1fe876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:22.527451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beadf86-dabd-422a-99a7-216f4a1fe876 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:22.527726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beadf86-dabd-422a-99a7-216f4a1fe876 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:22.528020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beadf86-dabd-422a-99a7-216f4a1fe876 None admin] PATH_INFO: `/v3/projects/7d5203564be545e1a1d8a33a58625555` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:22.529124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beadf86-dabd-422a-99a7-216f4a1fe876 None admin] RBAC: Authorizing `identity:get_project(project_id=7d5203564be545e1a1d8a33a58625555)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:22.530583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beadf86-dabd-422a-99a7-216f4a1fe876 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:22.532383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 161/330] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:29:22 2022] GET /identity/v3/projects/7d5203564be545e1a1d8a33a58625555 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:22.537628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beadf86-dabd-422a-99a7-216f4a1fe876 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:22.547996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 170/331] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:29:22.553905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55398d-7a70-46eb-9f6c-70117e3d6d43 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:22.617819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:22.638148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beadf86-dabd-422a-99a7-216f4a1fe876 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:22.654178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55398d-7a70-46eb-9f6c-70117e3d6d43 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:22.660092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:22.672083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36619b6-63b2-48e3-b50b-3e5b66986843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:22.672707 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36619b6-63b2-48e3-b50b-3e5b66986843 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:22.672988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36619b6-63b2-48e3-b50b-3e5b66986843 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:22.673256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36619b6-63b2-48e3-b50b-3e5b66986843 None admin] PATH_INFO: `/v3/groups/80584f1064a74e0a8b82e2dd91025db0` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:22.674361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36619b6-63b2-48e3-b50b-3e5b66986843 None admin] RBAC: Authorizing `identity:get_group(group_id=80584f1064a74e0a8b82e2dd91025db0)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:22.675778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36619b6-63b2-48e3-b50b-3e5b66986843 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:22.677333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 162/332] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Oct 24 13:29:22 2022] GET /identity/v3/groups/80584f1064a74e0a8b82e2dd91025db0 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:22.683784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36619b6-63b2-48e3-b50b-3e5b66986843 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:22.688712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9c69-ea1c-4c61-89aa-ade9a27ea5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:22.689346 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9c69-ea1c-4c61-89aa-ade9a27ea5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:22.689626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9c69-ea1c-4c61-89aa-ade9a27ea5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:22.689896 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9c69-ea1c-4c61-89aa-ade9a27ea5f6 None admin] PATH_INFO: `/v3/projects/73bbcd87686646c2a2c2fdd8f0e7bdd8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:22.691037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d9c69-ea1c-4c61-89aa-ade9a27ea5f6 None admin] RBAC: Authorizing `identity:get_project(project_id=73bbcd87686646c2a2c2fdd8f0e7bdd8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:22.692474 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d9c69-ea1c-4c61-89aa-ade9a27ea5f6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:22.694924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 171/333] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:29:22 2022] GET /identity/v3/projects/73bbcd87686646c2a2c2fdd8f0e7bdd8 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:22.700161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9c69-ea1c-4c61-89aa-ade9a27ea5f6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:22.784045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36619b6-63b2-48e3-b50b-3e5b66986843 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:22.796571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9c69-ea1c-4c61-89aa-ade9a27ea5f6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:22.817976 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd852dc9-23c0-4bd1-b75d-55e491f7e312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:22.819130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd852dc9-23c0-4bd1-b75d-55e491f7e312 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:22.819482 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd852dc9-23c0-4bd1-b75d-55e491f7e312 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:22.819760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd852dc9-23c0-4bd1-b75d-55e491f7e312 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:22.820473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd852dc9-23c0-4bd1-b75d-55e491f7e312 None admin] RBAC: Adding query filter params (group.id=80584f1064a74e0a8b82e2dd91025db0, role.id=f948e6f6ee4f42de92195aee35b05641, scope.project.id=7d5203564be545e1a1d8a33a58625555) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:22.820777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd852dc9-23c0-4bd1-b75d-55e491f7e312 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:22.822254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd852dc9-23c0-4bd1-b75d-55e491f7e312 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:22.827857 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 163/334] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Oct 24 13:29:22 2022] GET /identity/v3/role_assignments?group.id=80584f1064a74e0a8b82e2dd91025db0&scope.project.id=7d5203564be545e1a1d8a33a58625555&role.id=f948e6f6ee4f42de92195aee35b05641 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:22.829866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af89aa-d6dc-4940-bc20-87a070eb3eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:22.830526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af89aa-d6dc-4940-bc20-87a070eb3eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:22.830803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af89aa-d6dc-4940-bc20-87a070eb3eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:22.831072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af89aa-d6dc-4940-bc20-87a070eb3eb9 None admin] PATH_INFO: `/v3/groups/2846858e65d74896bf528feb5c6634a1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:22.832187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af89aa-d6dc-4940-bc20-87a070eb3eb9 None admin] RBAC: Authorizing `identity:get_group(group_id=2846858e65d74896bf528feb5c6634a1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:22.833621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af89aa-d6dc-4940-bc20-87a070eb3eb9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:22.835940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 172/335] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Oct 24 13:29:22 2022] GET /identity/v3/groups/2846858e65d74896bf528feb5c6634a1 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:22.842018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd852dc9-23c0-4bd1-b75d-55e491f7e312 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:22.900966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:22.947022 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd852dc9-23c0-4bd1-b75d-55e491f7e312 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:22.982636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e461a-4456-44bc-87a4-b5f01ea185e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:22.983246 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e461a-4456-44bc-87a4-b5f01ea185e8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:22.983520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e461a-4456-44bc-87a4-b5f01ea185e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:22.983783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e461a-4456-44bc-87a4-b5f01ea185e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:22.984505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e461a-4456-44bc-87a4-b5f01ea185e8 None admin] RBAC: Adding query filter params (group.id=2846858e65d74896bf528feb5c6634a1, role.id=8b32d66a3cb3483c8ede361ad8637f66, scope.project.id=73bbcd87686646c2a2c2fdd8f0e7bdd8) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:22.984803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e461a-4456-44bc-87a4-b5f01ea185e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:22.986242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e461a-4456-44bc-87a4-b5f01ea185e8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:22.992544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 164/336] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Oct 24 13:29:22 2022] GET /identity/v3/role_assignments?group.id=2846858e65d74896bf528feb5c6634a1&scope.project.id=73bbcd87686646c2a2c2fdd8f0e7bdd8&role.id=8b32d66a3cb3483c8ede361ad8637f66 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:25.581085 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec308b-d925-4882-88a6-1dc323f642ce None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:25.581625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec308b-d925-4882-88a6-1dc323f642ce None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:25.581625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec308b-d925-4882-88a6-1dc323f642ce None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:25.583280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 173/337] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:25.588871 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434f322-8bab-4438-b3d6-17c0fd1c4fdb None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:25.589310 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434f322-8bab-4438-b3d6-17c0fd1c4fdb None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:25.589394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434f322-8bab-4438-b3d6-17c0fd1c4fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:25.608771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e434f322-8bab-4438-b3d6-17c0fd1c4fdb None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:25.609945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e434f322-8bab-4438-b3d6-17c0fd1c4fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:25.703291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 165/338] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:29:26.023857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec308b-d925-4882-88a6-1dc323f642ce None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:26.028274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:26.038645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c5a71-869d-40f7-b413-0cc636d2bba3 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:26.039232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c5a71-869d-40f7-b413-0cc636d2bba3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:26.041503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c5a71-869d-40f7-b413-0cc636d2bba3 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:26.046028 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 166/339] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:26.051378 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370eb28-f6be-4351-9218-c604207fbe8d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:26.051683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370eb28-f6be-4351-9218-c604207fbe8d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:26.051934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370eb28-f6be-4351-9218-c604207fbe8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:26.087718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7370eb28-f6be-4351-9218-c604207fbe8d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:26.089001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7370eb28-f6be-4351-9218-c604207fbe8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:26.187920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:26.221969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec308b-d925-4882-88a6-1dc323f642ce None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:26.230376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 167/340] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:26.232352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb28c3-30c2-40b7-9146-72534859743c None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:26.232911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb28c3-30c2-40b7-9146-72534859743c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:26.233315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb28c3-30c2-40b7-9146-72534859743c None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:26.235317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 168/341] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:26.236458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbce689-a461-4f8d-8378-267f27cda638 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:26.236877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbce689-a461-4f8d-8378-267f27cda638 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:26.237279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbce689-a461-4f8d-8378-267f27cda638 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:26.238829 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 169/342] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:26.244696 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e57d24-d4c0-4f41-b73f-595c724e61e7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:26.246594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e57d24-d4c0-4f41-b73f-595c724e61e7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:26.247097 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e57d24-d4c0-4f41-b73f-595c724e61e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:26.266443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e7e57d24-d4c0-4f41-b73f-595c724e61e7 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:26.266666 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f24d2-0568-41fe-9a14-355a9b2ed65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:26.267354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f24d2-0568-41fe-9a14-355a9b2ed65a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:26.267702 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f24d2-0568-41fe-9a14-355a9b2ed65a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:26.268089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f24d2-0568-41fe-9a14-355a9b2ed65a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:26.268457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e7e57d24-d4c0-4f41-b73f-595c724e61e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:26.280237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f24d2-0568-41fe-9a14-355a9b2ed65a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:26.281791 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f24d2-0568-41fe-9a14-355a9b2ed65a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-e96f24d2-0568-41fe-9a14-355a9b2ed65a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:26.298060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:26.299259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:26.300338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:26.300338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:26.300338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:26.301392 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 174/343] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:26.302899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908773b-93ac-4665-8bd4-02150ef373d2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:26.303421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908773b-93ac-4665-8bd4-02150ef373d2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:26.303637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908773b-93ac-4665-8bd4-02150ef373d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:26.322884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e908773b-93ac-4665-8bd4-02150ef373d2 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:26.324259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e908773b-93ac-4665-8bd4-02150ef373d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:26.345437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 170/344] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:29:26.346790 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e57d24-d4c0-4f41-b73f-595c724e61e7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:26.407333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:26.437969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 175/345] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:26.454320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e57d24-d4c0-4f41-b73f-595c724e61e7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:26.465045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:26.491811 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281f676-d8d7-475b-9ab8-149bbccd0685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:26.492654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281f676-d8d7-475b-9ab8-149bbccd0685 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:26.493122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281f676-d8d7-475b-9ab8-149bbccd0685 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:26.493556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281f676-d8d7-475b-9ab8-149bbccd0685 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:26.494405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281f676-d8d7-475b-9ab8-149bbccd0685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:26.494901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281f676-d8d7-475b-9ab8-149bbccd0685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:26.496425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281f676-d8d7-475b-9ab8-149bbccd0685 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:26.514062 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 171/346] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:26.522380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908773b-93ac-4665-8bd4-02150ef373d2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:26.546423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281f676-d8d7-475b-9ab8-149bbccd0685 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:26.653045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281f676-d8d7-475b-9ab8-149bbccd0685 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:26.690224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff8db3-5cc9-492b-8180-1b9a26f56de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:26.691138 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff8db3-5cc9-492b-8180-1b9a26f56de7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:26.691606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff8db3-5cc9-492b-8180-1b9a26f56de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:26.692051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff8db3-5cc9-492b-8180-1b9a26f56de7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:26.703163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff8db3-5cc9-492b-8180-1b9a26f56de7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:26.704824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff8db3-5cc9-492b-8180-1b9a26f56de7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e4ff8db3-5cc9-492b-8180-1b9a26f56de7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:26.720777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:26.721709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:26.722591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:26.722591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:26.722591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:26.722751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908773b-93ac-4665-8bd4-02150ef373d2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:26.726195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 172/347] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:26.734625 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff8db3-5cc9-492b-8180-1b9a26f56de7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:26.775307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:26.788582 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af899cb-6b6a-411f-ab8f-a86e47ddf34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:26.789472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af899cb-6b6a-411f-ab8f-a86e47ddf34a None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:26.790257 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af899cb-6b6a-411f-ab8f-a86e47ddf34a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:26.791875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af899cb-6b6a-411f-ab8f-a86e47ddf34a None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:26.792708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af899cb-6b6a-411f-ab8f-a86e47ddf34a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:26.794340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af899cb-6b6a-411f-ab8f-a86e47ddf34a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:26.826636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 176/348] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:29:26 2022] POST /identity/v3/users => generated 246 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:26.827848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af899cb-6b6a-411f-ab8f-a86e47ddf34a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:26.880131 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff8db3-5cc9-492b-8180-1b9a26f56de7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:26.894260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:26.925139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a53ea-14bb-41df-94e3-049809e44dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:26.925805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a53ea-14bb-41df-94e3-049809e44dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:26.926013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a53ea-14bb-41df-94e3-049809e44dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:26.926334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a53ea-14bb-41df-94e3-049809e44dc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:26.927043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a53ea-14bb-41df-94e3-049809e44dc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:26.927375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a53ea-14bb-41df-94e3-049809e44dc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:26.929051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a53ea-14bb-41df-94e3-049809e44dc0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:26.945056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 173/349] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:26.945956 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a53ea-14bb-41df-94e3-049809e44dc0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:26.979202 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af899cb-6b6a-411f-ab8f-a86e47ddf34a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:27.019874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6af8de-1cf2-451d-82f4-1bfd592df17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:27.021482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6af8de-1cf2-451d-82f4-1bfd592df17f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:27.022090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6af8de-1cf2-451d-82f4-1bfd592df17f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:27.022090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6af8de-1cf2-451d-82f4-1bfd592df17f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:27.036141 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6af8de-1cf2-451d-82f4-1bfd592df17f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:27.037824 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6af8de-1cf2-451d-82f4-1bfd592df17f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-5a6af8de-1cf2-451d-82f4-1bfd592df17f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:27.059834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:27.060764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:27.061608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:27.061608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:27.061608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:27.061608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 177/350] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:27.063577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:27.067755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6af8de-1cf2-451d-82f4-1bfd592df17f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:27.098291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a53ea-14bb-41df-94e3-049809e44dc0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:27.142371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf29780-8b76-416d-9591-21ae63bd513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:27.144090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29780-8b76-416d-9591-21ae63bd513c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:27.144607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29780-8b76-416d-9591-21ae63bd513c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:27.145404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29780-8b76-416d-9591-21ae63bd513c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:27.163735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf29780-8b76-416d-9591-21ae63bd513c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:27.165631 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf29780-8b76-416d-9591-21ae63bd513c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:27.189139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-bcf29780-8b76-416d-9591-21ae63bd513c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:27.190834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:27.191925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:27.194007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:27.194007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:27.194007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:27.195865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 174/351] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:27.196855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf29780-8b76-416d-9591-21ae63bd513c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:27.204889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6af8de-1cf2-451d-82f4-1bfd592df17f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:27.241750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec494c-6598-4322-beb5-41c309e1d4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:27.242568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec494c-6598-4322-beb5-41c309e1d4d4 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:27.242938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec494c-6598-4322-beb5-41c309e1d4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:27.243279 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec494c-6598-4322-beb5-41c309e1d4d4 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:27.244019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec494c-6598-4322-beb5-41c309e1d4d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:27.245704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec494c-6598-4322-beb5-41c309e1d4d4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:27.268784 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 178/352] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:29:27 2022] POST /identity/v3/users => generated 248 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:27.269900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec494c-6598-4322-beb5-41c309e1d4d4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:27.326933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf29780-8b76-416d-9591-21ae63bd513c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:27.364914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:27.376020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c2ade-0656-4896-a496-ee031e13a84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:27.376848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c2ade-0656-4896-a496-ee031e13a84e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:27.377218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c2ade-0656-4896-a496-ee031e13a84e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:27.377596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c2ade-0656-4896-a496-ee031e13a84e None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:27.378627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c2ade-0656-4896-a496-ee031e13a84e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:27.378994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c2ade-0656-4896-a496-ee031e13a84e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:27.381914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c2ade-0656-4896-a496-ee031e13a84e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:27.399667 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec494c-6598-4322-beb5-41c309e1d4d4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:27.399881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 175/353] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:27.401247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b85b-90d8-416f-baa0-9c3c9abb3993 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:27.401616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b85b-90d8-416f-baa0-9c3c9abb3993 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:27.401933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b85b-90d8-416f-baa0-9c3c9abb3993 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:27.404662 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 176/354] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:27.405474 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45b85b-90d8-416f-baa0-9c3c9abb3993 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:27.441442 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85392fd-c5ad-4aa1-a724-0c62ca04220f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:27.442428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85392fd-c5ad-4aa1-a724-0c62ca04220f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:27.442846 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85392fd-c5ad-4aa1-a724-0c62ca04220f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:27.443996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85392fd-c5ad-4aa1-a724-0c62ca04220f None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:27.445340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85392fd-c5ad-4aa1-a724-0c62ca04220f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:27.445796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85392fd-c5ad-4aa1-a724-0c62ca04220f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:27.447411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85392fd-c5ad-4aa1-a724-0c62ca04220f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:27.462024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 179/355] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:27.467764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0915ad7-7e17-4e16-8c0b-49a97b9c719b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:27.467764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0915ad7-7e17-4e16-8c0b-49a97b9c719b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:27.467764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0915ad7-7e17-4e16-8c0b-49a97b9c719b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:27.491914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d0915ad7-7e17-4e16-8c0b-49a97b9c719b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:27.493405 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d0915ad7-7e17-4e16-8c0b-49a97b9c719b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:27.497475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:27.547619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45b85b-90d8-416f-baa0-9c3c9abb3993 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:27.584862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 180/356] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:27.585366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0915ad7-7e17-4e16-8c0b-49a97b9c719b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:27.587375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220982d-2eba-4cc6-a920-1cb4a032bc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:27.588311 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220982d-2eba-4cc6-a920-1cb4a032bc6a None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:27.588854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220982d-2eba-4cc6-a920-1cb4a032bc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:27.589176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220982d-2eba-4cc6-a920-1cb4a032bc6a None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:27.589885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220982d-2eba-4cc6-a920-1cb4a032bc6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:27.591416 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220982d-2eba-4cc6-a920-1cb4a032bc6a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:27.623810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 177/357] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:29:27 2022] POST /identity/v3/users => generated 249 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:27.735087 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0915ad7-7e17-4e16-8c0b-49a97b9c719b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:27.773459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc4890-3a28-4f16-8951-d00e8fc11299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:27.774097 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc4890-3a28-4f16-8951-d00e8fc11299 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:27.774490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc4890-3a28-4f16-8951-d00e8fc11299 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:27.774764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc4890-3a28-4f16-8951-d00e8fc11299 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:27.775435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc4890-3a28-4f16-8951-d00e8fc11299 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:27.776906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc4890-3a28-4f16-8951-d00e8fc11299 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:27.789259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:27.803837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 181/358] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:29:27 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:27.991795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220982d-2eba-4cc6-a920-1cb4a032bc6a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:28.045880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:28.103948 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220982d-2eba-4cc6-a920-1cb4a032bc6a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:28.140263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b1d88-70c1-4c23-ac13-7b56b4cbf795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:28.140881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b1d88-70c1-4c23-ac13-7b56b4cbf795 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:28.141159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b1d88-70c1-4c23-ac13-7b56b4cbf795 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:28.141433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b1d88-70c1-4c23-ac13-7b56b4cbf795 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:28.142104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b1d88-70c1-4c23-ac13-7b56b4cbf795 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:28.143500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b1d88-70c1-4c23-ac13-7b56b4cbf795 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:28.159183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.sql.core [None req-883b1d88-70c1-4c23-ac13-7b56b4cbf795 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 13:29:28.159183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 24 13:29:28.159183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [parameters: {'id': '87ed041afb9d4070b077e45fb378d777', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 24 13:29:28.159183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81688) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-883b1d88-70c1-4c23-ac13-7b56b4cbf795 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result.read() Oct 24 13:29:28.175379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return ref.to_dict() Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:29:28.176278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application yield resource Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise self.value Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application session.commit() Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:29:28.177206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:29:28.179078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:29:28.180111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result.read() Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:29:28.181092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [parameters: {'id': '87ed041afb9d4070b077e45fb378d777', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:28.183240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 24 13:29:28.185319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:28.187394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 178/359] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Oct 24 13:29:27 2022] POST /identity/v3/roles => generated 144 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:28.187606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc4890-3a28-4f16-8951-d00e8fc11299 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:28.291819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc4890-3a28-4f16-8951-d00e8fc11299 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:28.327523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfab06-82d1-499a-a47e-31ca6112006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:28.328166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfab06-82d1-499a-a47e-31ca6112006f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:28.328493 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfab06-82d1-499a-a47e-31ca6112006f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:28.328771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfab06-82d1-499a-a47e-31ca6112006f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:28.340414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfab06-82d1-499a-a47e-31ca6112006f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b2dfab06-82d1-499a-a47e-31ca6112006f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:28.345823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:28.346723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:28.347507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 182/360] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:29:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:28.354271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b1d88-70c1-4c23-ac13-7b56b4cbf795 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:28.477957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b1d88-70c1-4c23-ac13-7b56b4cbf795 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:28.492218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:28.515582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa3555-7420-4faa-84e3-3ee020043451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:28.516529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3555-7420-4faa-84e3-3ee020043451 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:28.517010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3555-7420-4faa-84e3-3ee020043451 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:28.517551 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3555-7420-4faa-84e3-3ee020043451 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:28.518437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa3555-7420-4faa-84e3-3ee020043451 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:28.518931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa3555-7420-4faa-84e3-3ee020043451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:28.520521 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa3555-7420-4faa-84e3-3ee020043451 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:28.529136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 179/361] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:29.294420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa5d2f-9ad5-4024-aa7f-b095ef1ee6fa None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:29.294923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa5d2f-9ad5-4024-aa7f-b095ef1ee6fa None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:29.294994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa5d2f-9ad5-4024-aa7f-b095ef1ee6fa None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:29.296578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 183/362] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:29.301463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c7022-0c24-40ba-8746-7af98086be30 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:29.301871 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c7022-0c24-40ba-8746-7af98086be30 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:29.302033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c7022-0c24-40ba-8746-7af98086be30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:29.328740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-745c7022-0c24-40ba-8746-7af98086be30 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:29.329948 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-745c7022-0c24-40ba-8746-7af98086be30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:29.419771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 180/363] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:29.792254 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92051e9e-879f-4a2d-b027-0c09b8ae9715 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:29.792254 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92051e9e-879f-4a2d-b027-0c09b8ae9715 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:29.792254 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92051e9e-879f-4a2d-b027-0c09b8ae9715 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:29.793005 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 184/364] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:29.798362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea4cd1-7373-4cb7-a752-018d74d0c2c6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:29.798892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea4cd1-7373-4cb7-a752-018d74d0c2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:29.799366 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea4cd1-7373-4cb7-a752-018d74d0c2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:29.820652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-51ea4cd1-7373-4cb7-a752-018d74d0c2c6 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:29.822189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-51ea4cd1-7373-4cb7-a752-018d74d0c2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:29.828070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92051e9e-879f-4a2d-b027-0c09b8ae9715 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:29.924430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 181/365] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:29.947221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:30.017887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92051e9e-879f-4a2d-b027-0c09b8ae9715 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:30.068754 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c89b54-0ed5-458d-b98d-d1d47d5e0211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:30.069397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c89b54-0ed5-458d-b98d-d1d47d5e0211 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:30.069677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c89b54-0ed5-458d-b98d-d1d47d5e0211 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:30.069949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c89b54-0ed5-458d-b98d-d1d47d5e0211 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:30.083282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c89b54-0ed5-458d-b98d-d1d47d5e0211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-c4c89b54-0ed5-458d-b98d-d1d47d5e0211 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:30.089000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:30.089863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:30.090673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 185/366] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:30.095891 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea4cd1-7373-4cb7-a752-018d74d0c2c6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:30.142317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:30.213063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea4cd1-7373-4cb7-a752-018d74d0c2c6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:30.247846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62a0cb-c80e-42ba-b0e8-2bce47050e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:30.248456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62a0cb-c80e-42ba-b0e8-2bce47050e68 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:30.248726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62a0cb-c80e-42ba-b0e8-2bce47050e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:30.248989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62a0cb-c80e-42ba-b0e8-2bce47050e68 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:30.249641 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62a0cb-c80e-42ba-b0e8-2bce47050e68 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:30.249934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62a0cb-c80e-42ba-b0e8-2bce47050e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:30.251317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62a0cb-c80e-42ba-b0e8-2bce47050e68 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:30.256762 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c89b54-0ed5-458d-b98d-d1d47d5e0211 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:30.260590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 182/367] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:30.267136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62a0cb-c80e-42ba-b0e8-2bce47050e68 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:30.381914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:30.389932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62a0cb-c80e-42ba-b0e8-2bce47050e68 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:30.448341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d4f75-99a5-4db0-b695-0c3a50977c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:30.449144 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d4f75-99a5-4db0-b695-0c3a50977c80 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:30.449420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d4f75-99a5-4db0-b695-0c3a50977c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:30.449684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d4f75-99a5-4db0-b695-0c3a50977c80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:30.461617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d4f75-99a5-4db0-b695-0c3a50977c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:30.463144 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d4f75-99a5-4db0-b695-0c3a50977c80 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:30.476152 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c89b54-0ed5-458d-b98d-d1d47d5e0211 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-294d4f75-99a5-4db0-b695-0c3a50977c80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:30.479421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:30.480381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:30.481220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:30.481220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:30.481220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:30.481220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 183/368] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:30.481720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48851a-82d7-4738-83f7-7054146e0f78 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:30.481989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48851a-82d7-4738-83f7-7054146e0f78 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:30.482302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48851a-82d7-4738-83f7-7054146e0f78 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:30.486202 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 184/369] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:30.491299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48851a-82d7-4738-83f7-7054146e0f78 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:30.497529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:30.517914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e6f14-ace2-4fe6-a066-537a868934ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:30.518695 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e6f14-ace2-4fe6-a066-537a868934ac None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:30.519046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e6f14-ace2-4fe6-a066-537a868934ac None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:30.519391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e6f14-ace2-4fe6-a066-537a868934ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:30.533840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e6f14-ace2-4fe6-a066-537a868934ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-c02e6f14-ace2-4fe6-a066-537a868934ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:30.541222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:30.542128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:30.542973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:30.546496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 186/370] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:30.547456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed1378-615c-41ad-bd77-ab28a6123541 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:30.547740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed1378-615c-41ad-bd77-ab28a6123541 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:30.548006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed1378-615c-41ad-bd77-ab28a6123541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:30.572623 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4eed1378-615c-41ad-bd77-ab28a6123541 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:30.574125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4eed1378-615c-41ad-bd77-ab28a6123541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:30.621389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48851a-82d7-4738-83f7-7054146e0f78 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:30.651421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 187/371] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:30.652742 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed1378-615c-41ad-bd77-ab28a6123541 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:30.673547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723edda4-2cc7-4774-be59-ed2bb3ec2f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:30.674572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723edda4-2cc7-4774-be59-ed2bb3ec2f66 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:30.675076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723edda4-2cc7-4774-be59-ed2bb3ec2f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:30.675573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723edda4-2cc7-4774-be59-ed2bb3ec2f66 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:30.676412 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723edda4-2cc7-4774-be59-ed2bb3ec2f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:30.676923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723edda4-2cc7-4774-be59-ed2bb3ec2f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:30.680069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723edda4-2cc7-4774-be59-ed2bb3ec2f66 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:30.706261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 185/372] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:30.712307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723edda4-2cc7-4774-be59-ed2bb3ec2f66 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:30.731057 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:30.846956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed1378-615c-41ad-bd77-ab28a6123541 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:30.863719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:30.906518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723edda4-2cc7-4774-be59-ed2bb3ec2f66 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:30.908623 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55350d6e-1e17-42a1-9032-d565e5b94aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:30.909339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350d6e-1e17-42a1-9032-d565e5b94aac None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:30.910320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350d6e-1e17-42a1-9032-d565e5b94aac None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:30.910821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350d6e-1e17-42a1-9032-d565e5b94aac None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:30.911522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55350d6e-1e17-42a1-9032-d565e5b94aac None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:30.912377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55350d6e-1e17-42a1-9032-d565e5b94aac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:30.913763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55350d6e-1e17-42a1-9032-d565e5b94aac None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:30.923035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 188/373] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:30.929056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c14786-684d-42b9-a9bb-8de7f675d815 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:30.929314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c14786-684d-42b9-a9bb-8de7f675d815 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:30.929591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c14786-684d-42b9-a9bb-8de7f675d815 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:30.935045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 189/374] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:30.935680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c14786-684d-42b9-a9bb-8de7f675d815 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:30.960375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a7bd2-5d9b-427b-93b9-bee898d7190e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:30.961167 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7bd2-5d9b-427b-93b9-bee898d7190e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:30.961441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7bd2-5d9b-427b-93b9-bee898d7190e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:30.961706 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7bd2-5d9b-427b-93b9-bee898d7190e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:30.968944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a7bd2-5d9b-427b-93b9-bee898d7190e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:30.970394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a7bd2-5d9b-427b-93b9-bee898d7190e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-381a7bd2-5d9b-427b-93b9-bee898d7190e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:30.983251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:30.984207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:30.985102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:30.985102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:30.985102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:30.985102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:30.985102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:30.985102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:30.985102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:30.985102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:30.985102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:29:30.985102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:30.985102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 186/375] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Oct 24 13:29:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:30.990427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686668b-229e-43c4-b82d-de0e89ca21a5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:30.990712 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686668b-229e-43c4-b82d-de0e89ca21a5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:30.990985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686668b-229e-43c4-b82d-de0e89ca21a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.010131 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c686668b-229e-43c4-b82d-de0e89ca21a5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:31.011357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c686668b-229e-43c4-b82d-de0e89ca21a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:31.050510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c14786-684d-42b9-a9bb-8de7f675d815 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:31.076639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:31.094201 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 187/376] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:31.094201 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686668b-229e-43c4-b82d-de0e89ca21a5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:31.096132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8a402-bb24-47d8-8d95-036153f67d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:31.096777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8a402-bb24-47d8-8d95-036153f67d40 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.097055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8a402-bb24-47d8-8d95-036153f67d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.097316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8a402-bb24-47d8-8d95-036153f67d40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.109810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8a402-bb24-47d8-8d95-036153f67d40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:31.111356 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8a402-bb24-47d8-8d95-036153f67d40 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-23f8a402-bb24-47d8-8d95-036153f67d40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:31.128953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:31.129866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:31.130801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:31.130801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:31.130801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:31.133006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 190/377] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:31.134094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8a402-bb24-47d8-8d95-036153f67d40 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:31.163142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:31.232539 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686668b-229e-43c4-b82d-de0e89ca21a5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:31.269122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed31ff-2037-49a2-b5d1-73b8f12f7fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:31.269760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed31ff-2037-49a2-b5d1-73b8f12f7fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.270213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed31ff-2037-49a2-b5d1-73b8f12f7fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.270444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed31ff-2037-49a2-b5d1-73b8f12f7fe5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.274940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8a402-bb24-47d8-8d95-036153f67d40 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:31.288061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed31ff-2037-49a2-b5d1-73b8f12f7fe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6bed31ff-2037-49a2-b5d1-73b8f12f7fe5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:31.294252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:31.295249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:31.296056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 188/378] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:31.296726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed31ff-2037-49a2-b5d1-73b8f12f7fe5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:31.325204 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c37a6-6bfd-4d90-936f-5b4284d9b8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:31.326276 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c37a6-6bfd-4d90-936f-5b4284d9b8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.327455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c37a6-6bfd-4d90-936f-5b4284d9b8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.327981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c37a6-6bfd-4d90-936f-5b4284d9b8c9 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.329343 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c37a6-6bfd-4d90-936f-5b4284d9b8c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:31.330148 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c37a6-6bfd-4d90-936f-5b4284d9b8c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:31.332703 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c37a6-6bfd-4d90-936f-5b4284d9b8c9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:31.370890 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:31.392435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 191/379] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Oct 24 13:29:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:31.394067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c37a6-6bfd-4d90-936f-5b4284d9b8c9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:31.416161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed31ff-2037-49a2-b5d1-73b8f12f7fe5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:31.450248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e139bdc-4098-4d8d-855c-3882989f8cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:31.450851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e139bdc-4098-4d8d-855c-3882989f8cee None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.451124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e139bdc-4098-4d8d-855c-3882989f8cee None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.451393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e139bdc-4098-4d8d-855c-3882989f8cee None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.452022 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e139bdc-4098-4d8d-855c-3882989f8cee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:31.452318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e139bdc-4098-4d8d-855c-3882989f8cee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:31.453685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e139bdc-4098-4d8d-855c-3882989f8cee None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:31.465719 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 189/380] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:31.466547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e139bdc-4098-4d8d-855c-3882989f8cee None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:31.481938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:31.504560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c37a6-6bfd-4d90-936f-5b4284d9b8c9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:31.538091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92711aad-5ffb-45f5-92b9-f41fd3e8c5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:31.538718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92711aad-5ffb-45f5-92b9-f41fd3e8c5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.539000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92711aad-5ffb-45f5-92b9-f41fd3e8c5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.539270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92711aad-5ffb-45f5-92b9-f41fd3e8c5a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.539926 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92711aad-5ffb-45f5-92b9-f41fd3e8c5a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:31.540231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92711aad-5ffb-45f5-92b9-f41fd3e8c5a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:31.541851 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92711aad-5ffb-45f5-92b9-f41fd3e8c5a4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:31.549545 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 192/381] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:31.550354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92711aad-5ffb-45f5-92b9-f41fd3e8c5a4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:31.567019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e139bdc-4098-4d8d-855c-3882989f8cee None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:31.583835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:31.600783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5e78d-14a8-400e-829f-8bbb334724ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:31.601402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5e78d-14a8-400e-829f-8bbb334724ef None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.601677 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5e78d-14a8-400e-829f-8bbb334724ef None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.601941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5e78d-14a8-400e-829f-8bbb334724ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.611917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5e78d-14a8-400e-829f-8bbb334724ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-18b5e78d-14a8-400e-829f-8bbb334724ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:31.617694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:31.618821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 190/382] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:31.624679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa4cd-e9c6-4b6d-bcc4-29a432ea7575 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.625384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa4cd-e9c6-4b6d-bcc4-29a432ea7575 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.625384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fa4cd-e9c6-4b6d-bcc4-29a432ea7575 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.625384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 191/383] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:31.625384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fa4cd-e9c6-4b6d-bcc4-29a432ea7575 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:31.655026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92711aad-5ffb-45f5-92b9-f41fd3e8c5a4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:31.688707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9152989-a42d-4726-9ecb-eae365470a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:31.689269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152989-a42d-4726-9ecb-eae365470a68 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.689572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152989-a42d-4726-9ecb-eae365470a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.689838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152989-a42d-4726-9ecb-eae365470a68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.700063 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:31.702494 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9152989-a42d-4726-9ecb-eae365470a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:31.704627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9152989-a42d-4726-9ecb-eae365470a68 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-a9152989-a42d-4726-9ecb-eae365470a68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:31.718771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:31.719705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:31.720551 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:31.720551 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:31.720551 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:31.720551 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 193/384] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:31.720761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9152989-a42d-4726-9ecb-eae365470a68 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:31.729127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fa4cd-e9c6-4b6d-bcc4-29a432ea7575 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:31.763080 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed6c6d-b38b-452f-aafd-eb10783136f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:31.763701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed6c6d-b38b-452f-aafd-eb10783136f0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.763979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed6c6d-b38b-452f-aafd-eb10783136f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.764250 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed6c6d-b38b-452f-aafd-eb10783136f0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.771542 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed6c6d-b38b-452f-aafd-eb10783136f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:31.772953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed6c6d-b38b-452f-aafd-eb10783136f0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-78ed6c6d-b38b-452f-aafd-eb10783136f0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:31.783758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:31.784732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:31.785619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:31.785619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:31.785619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:31.785619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:31.785619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:31.785619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:31.785619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:31.785619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:31.785619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:29:31.785619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:31.785619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 192/385] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:31.786210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed6c6d-b38b-452f-aafd-eb10783136f0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:31.821676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9152989-a42d-4726-9ecb-eae365470a68 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:31.830036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:31.855416 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6ffab-8527-4d9c-9787-8baf0403931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:31.856045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6ffab-8527-4d9c-9787-8baf0403931e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.856329 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6ffab-8527-4d9c-9787-8baf0403931e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.856604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6ffab-8527-4d9c-9787-8baf0403931e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.867490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6ffab-8527-4d9c-9787-8baf0403931e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:31.868918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6ffab-8527-4d9c-9787-8baf0403931e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f1b6ffab-8527-4d9c-9787-8baf0403931e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:31.883215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:31.884215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:31.885053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:31.885053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:31.885053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:31.885053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 194/386] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:31.885529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b24b46-9fd6-4337-98ee-7745342b651e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.885767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b24b46-9fd6-4337-98ee-7745342b651e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.886035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b24b46-9fd6-4337-98ee-7745342b651e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.894423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed6c6d-b38b-452f-aafd-eb10783136f0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:31.903551 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-88b24b46-9fd6-4337-98ee-7745342b651e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:31.904913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-88b24b46-9fd6-4337-98ee-7745342b651e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:31.929887 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbffbdd-b6b3-4e53-ac72-75ce02d91bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:31.930518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbffbdd-b6b3-4e53-ac72-75ce02d91bda None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:31.930798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbffbdd-b6b3-4e53-ac72-75ce02d91bda None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:31.931070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbffbdd-b6b3-4e53-ac72-75ce02d91bda None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:31.931728 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbffbdd-b6b3-4e53-ac72-75ce02d91bda None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:31.932030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbffbdd-b6b3-4e53-ac72-75ce02d91bda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:31.933382 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbffbdd-b6b3-4e53-ac72-75ce02d91bda None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:31.941213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 193/387] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:31.941916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbffbdd-b6b3-4e53-ac72-75ce02d91bda None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:31.945044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:31.972312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 195/388] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:31.973141 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b24b46-9fd6-4337-98ee-7745342b651e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.059193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:32.062582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbffbdd-b6b3-4e53-ac72-75ce02d91bda None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.083300 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b24b46-9fd6-4337-98ee-7745342b651e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.098179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a4610-e270-400f-9d7e-193834381e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.098790 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a4610-e270-400f-9d7e-193834381e31 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.099549 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a4610-e270-400f-9d7e-193834381e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.099825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a4610-e270-400f-9d7e-193834381e31 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.100458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a4610-e270-400f-9d7e-193834381e31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:32.100759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a4610-e270-400f-9d7e-193834381e31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.102148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a4610-e270-400f-9d7e-193834381e31 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.114363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 194/389] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:32.115951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a4610-e270-400f-9d7e-193834381e31 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.124480 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc038e9-e95d-408f-bc3b-648bef4aa603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.125091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc038e9-e95d-408f-bc3b-648bef4aa603 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.125371 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc038e9-e95d-408f-bc3b-648bef4aa603 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.125648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc038e9-e95d-408f-bc3b-648bef4aa603 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.126434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc038e9-e95d-408f-bc3b-648bef4aa603 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:32.126740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc038e9-e95d-408f-bc3b-648bef4aa603 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.128189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc038e9-e95d-408f-bc3b-648bef4aa603 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.145997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 196/390] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Oct 24 13:29:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:32.146877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc038e9-e95d-408f-bc3b-648bef4aa603 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.182120 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:32.235128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a4610-e270-400f-9d7e-193834381e31 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.258037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc038e9-e95d-408f-bc3b-648bef4aa603 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.268084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb89a16-8531-4308-994d-14dfed376b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.268687 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb89a16-8531-4308-994d-14dfed376b2d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.268966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb89a16-8531-4308-994d-14dfed376b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.269232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb89a16-8531-4308-994d-14dfed376b2d None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.269861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb89a16-8531-4308-994d-14dfed376b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:32.270159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb89a16-8531-4308-994d-14dfed376b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.272276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb89a16-8531-4308-994d-14dfed376b2d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.283186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 195/391] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:32.283960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb89a16-8531-4308-994d-14dfed376b2d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.292768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51095c40-6204-45a1-868a-312710c0bbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.293400 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51095c40-6204-45a1-868a-312710c0bbdc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.293677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51095c40-6204-45a1-868a-312710c0bbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.293945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51095c40-6204-45a1-868a-312710c0bbdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.303892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51095c40-6204-45a1-868a-312710c0bbdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.305323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51095c40-6204-45a1-868a-312710c0bbdc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.307328 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-51095c40-6204-45a1-868a-312710c0bbdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:32.320934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:32.321848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:32.322735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:32.322735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:32.322735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:32.322735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 197/392] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:32.322944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51095c40-6204-45a1-868a-312710c0bbdc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.391448 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb89a16-8531-4308-994d-14dfed376b2d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.410024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:32.426627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86b2fa-b114-4a64-b186-79157c719c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.427181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51095c40-6204-45a1-868a-312710c0bbdc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.427320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86b2fa-b114-4a64-b186-79157c719c8a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.427757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86b2fa-b114-4a64-b186-79157c719c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.428031 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86b2fa-b114-4a64-b186-79157c719c8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.434779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86b2fa-b114-4a64-b186-79157c719c8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.436221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86b2fa-b114-4a64-b186-79157c719c8a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-7f86b2fa-b114-4a64-b186-79157c719c8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:32.447988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:32.448851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:32.449695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:32.449695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:32.449695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:32.449695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:32.449695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:32.449695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:32.449695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:32.449695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:32.449695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 196/393] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:32.450193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86b2fa-b114-4a64-b186-79157c719c8a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.461534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c3d3b-5bee-434c-8913-146936326672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.462105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c3d3b-5bee-434c-8913-146936326672 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.462415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c3d3b-5bee-434c-8913-146936326672 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.462684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c3d3b-5bee-434c-8913-146936326672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.473010 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c3d3b-5bee-434c-8913-146936326672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.475043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c3d3b-5bee-434c-8913-146936326672 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f04c3d3b-5bee-434c-8913-146936326672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:32.489004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:32.490676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:32.491694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:32.491694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:32.491694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:32.491694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 198/394] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:32.491940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c3d3b-5bee-434c-8913-146936326672 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.532794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:32.554159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86b2fa-b114-4a64-b186-79157c719c8a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.589194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c94ff9-c5f1-48e5-ba09-592e9ff51d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.589977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c94ff9-c5f1-48e5-ba09-592e9ff51d7d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.590286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c94ff9-c5f1-48e5-ba09-592e9ff51d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.590382 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c3d3b-5bee-434c-8913-146936326672 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.590558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c94ff9-c5f1-48e5-ba09-592e9ff51d7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.601349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c94ff9-c5f1-48e5-ba09-592e9ff51d7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.602914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c94ff9-c5f1-48e5-ba09-592e9ff51d7d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-94c94ff9-c5f1-48e5-ba09-592e9ff51d7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:32.617421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:32.618541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:32.619428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:32.619428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:32.619428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:32.619428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 197/395] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:32.619718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c94ff9-c5f1-48e5-ba09-592e9ff51d7d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.625266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948ed6d-88ef-4ba7-82b9-77f1e1ad7fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.625892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948ed6d-88ef-4ba7-82b9-77f1e1ad7fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.626188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948ed6d-88ef-4ba7-82b9-77f1e1ad7fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.626464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948ed6d-88ef-4ba7-82b9-77f1e1ad7fd8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.629175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:32.635994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948ed6d-88ef-4ba7-82b9-77f1e1ad7fd8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.637535 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948ed6d-88ef-4ba7-82b9-77f1e1ad7fd8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b948ed6d-88ef-4ba7-82b9-77f1e1ad7fd8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:32.650717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:32.651691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:32.652938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:32.652938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:32.652938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:32.652938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:32.652938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:32.652938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:32.652938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:32.652938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:32.652938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:29:32.652938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:32.652938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 199/396] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:32.657102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948ed6d-88ef-4ba7-82b9-77f1e1ad7fd8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.719681 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c94ff9-c5f1-48e5-ba09-592e9ff51d7d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.748724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:32.754372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6852d3d-0eef-46a6-901d-3aff4215ec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.754977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6852d3d-0eef-46a6-901d-3aff4215ec38 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.755270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6852d3d-0eef-46a6-901d-3aff4215ec38 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.755709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6852d3d-0eef-46a6-901d-3aff4215ec38 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.756316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6852d3d-0eef-46a6-901d-3aff4215ec38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:32.756615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6852d3d-0eef-46a6-901d-3aff4215ec38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.757967 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6852d3d-0eef-46a6-901d-3aff4215ec38 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.758635 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948ed6d-88ef-4ba7-82b9-77f1e1ad7fd8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.769385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 198/397] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:32.771223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6852d3d-0eef-46a6-901d-3aff4215ec38 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.793484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217369d-f416-4574-ba61-cb2fdc72c41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.794100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217369d-f416-4574-ba61-cb2fdc72c41f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.794412 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217369d-f416-4574-ba61-cb2fdc72c41f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.794688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1217369d-f416-4574-ba61-cb2fdc72c41f None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.795374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217369d-f416-4574-ba61-cb2fdc72c41f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:32.795678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217369d-f416-4574-ba61-cb2fdc72c41f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.797113 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1217369d-f416-4574-ba61-cb2fdc72c41f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.815405 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 200/398] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:32.816219 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217369d-f416-4574-ba61-cb2fdc72c41f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.871079 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:32.875978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6852d3d-0eef-46a6-901d-3aff4215ec38 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.911266 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380d1bb-946d-4f0c-af8e-ef9ad6924600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.912085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380d1bb-946d-4f0c-af8e-ef9ad6924600 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.912371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380d1bb-946d-4f0c-af8e-ef9ad6924600 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.912641 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380d1bb-946d-4f0c-af8e-ef9ad6924600 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.913280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380d1bb-946d-4f0c-af8e-ef9ad6924600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:32.913582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380d1bb-946d-4f0c-af8e-ef9ad6924600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.914623 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1217369d-f416-4574-ba61-cb2fdc72c41f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:32.914982 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380d1bb-946d-4f0c-af8e-ef9ad6924600 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.926146 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 199/399] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:32.926952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380d1bb-946d-4f0c-af8e-ef9ad6924600 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.948581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f0ec6-e24a-4bac-91cd-1e759eeea6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:32.949193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f0ec6-e24a-4bac-91cd-1e759eeea6de None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:32.949472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f0ec6-e24a-4bac-91cd-1e759eeea6de None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:32.949740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f0ec6-e24a-4bac-91cd-1e759eeea6de None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:32.950401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f0ec6-e24a-4bac-91cd-1e759eeea6de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:32.950715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f0ec6-e24a-4bac-91cd-1e759eeea6de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:32.952075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f0ec6-e24a-4bac-91cd-1e759eeea6de None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:32.963471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 201/400] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:32.964251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f0ec6-e24a-4bac-91cd-1e759eeea6de None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:32.979394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:33.026782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380d1bb-946d-4f0c-af8e-ef9ad6924600 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:33.060045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99d4ea-ca7d-42a3-9af8-bfaf384fe835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:33.060646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99d4ea-ca7d-42a3-9af8-bfaf384fe835 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:33.060926 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99d4ea-ca7d-42a3-9af8-bfaf384fe835 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:33.061197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99d4ea-ca7d-42a3-9af8-bfaf384fe835 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:33.061942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99d4ea-ca7d-42a3-9af8-bfaf384fe835 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:33.062268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f0ec6-e24a-4bac-91cd-1e759eeea6de None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:33.062366 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99d4ea-ca7d-42a3-9af8-bfaf384fe835 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:33.063714 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99d4ea-ca7d-42a3-9af8-bfaf384fe835 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:33.078764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:33.096160 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c0985-d6c1-4651-ae58-4dc455a93a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:33.096777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c0985-d6c1-4651-ae58-4dc455a93a7b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:33.097050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c0985-d6c1-4651-ae58-4dc455a93a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:33.097321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c0985-d6c1-4651-ae58-4dc455a93a7b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:33.100683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 200/401] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:33.101649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99d4ea-ca7d-42a3-9af8-bfaf384fe835 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:33.106652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c0985-d6c1-4651-ae58-4dc455a93a7b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:33.108088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c0985-d6c1-4651-ae58-4dc455a93a7b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-7c5c0985-d6c1-4651-ae58-4dc455a93a7b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:33.119899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:33.120969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 202/402] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:33.126598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c0985-d6c1-4651-ae58-4dc455a93a7b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:33.207043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99d4ea-ca7d-42a3-9af8-bfaf384fe835 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:33.219134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:33.222607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c0985-d6c1-4651-ae58-4dc455a93a7b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:33.243823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0ba8c-ee33-45cc-a83e-f45b96a0560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:33.244403 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ba8c-ee33-45cc-a83e-f45b96a0560b None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:33.244679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ba8c-ee33-45cc-a83e-f45b96a0560b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:33.244948 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ba8c-ee33-45cc-a83e-f45b96a0560b None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/18bb93669846435a85eb34771cfb09fe/roles/037adbca67514eda861c68218a60c50a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:33.257317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207a420-d2a7-48a0-8158-e4b209230c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:33.257939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207a420-d2a7-48a0-8158-e4b209230c06 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:33.258253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207a420-d2a7-48a0-8158-e4b209230c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:33.258521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207a420-d2a7-48a0-8158-e4b209230c06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:33.259368 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0ba8c-ee33-45cc-a83e-f45b96a0560b None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=18bb93669846435a85eb34771cfb09fe, role_id=037adbca67514eda861c68218a60c50a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:33.260802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0ba8c-ee33-45cc-a83e-f45b96a0560b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:33.265225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207a420-d2a7-48a0-8158-e4b209230c06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:33.266889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207a420-d2a7-48a0-8158-e4b209230c06 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:33.274489 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 201/403] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:29:33 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/18bb93669846435a85eb34771cfb09fe/roles/037adbca67514eda861c68218a60c50a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:33.275030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0ba8c-ee33-45cc-a83e-f45b96a0560b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f207a420-d2a7-48a0-8158-e4b209230c06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:33.279647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:33.280589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:33.281450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:33.281450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:33.281450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:33.281450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:33.281450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:33.281450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:33.281450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:33.281450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:33.281450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 203/404] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:33.282002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207a420-d2a7-48a0-8158-e4b209230c06 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:33.323238 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:33.401602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207a420-d2a7-48a0-8158-e4b209230c06 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:33.408535 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0ba8c-ee33-45cc-a83e-f45b96a0560b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:33.442518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0705d0-0a39-4eed-8131-c273b1b0e56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:33.443697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0705d0-0a39-4eed-8131-c273b1b0e56a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:33.443977 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0705d0-0a39-4eed-8131-c273b1b0e56a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:33.444247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0705d0-0a39-4eed-8131-c273b1b0e56a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:33.447210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a889f-3cbd-4643-9819-3cbc9664e998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:33.447826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a889f-3cbd-4643-9819-3cbc9664e998 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:33.448106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a889f-3cbd-4643-9819-3cbc9664e998 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:33.448375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a889f-3cbd-4643-9819-3cbc9664e998 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:33.449024 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a889f-3cbd-4643-9819-3cbc9664e998 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:33.450561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a889f-3cbd-4643-9819-3cbc9664e998 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:33.456974 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0705d0-0a39-4eed-8131-c273b1b0e56a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:33.458535 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0705d0-0a39-4eed-8131-c273b1b0e56a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-5c0705d0-0a39-4eed-8131-c273b1b0e56a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:33.475634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:33.477095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:33.477936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:33.477936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:33.477936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:33.478104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 202/405] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:29:33 2022] POST /identity/v3/users => generated 247 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:33.478104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a889f-3cbd-4643-9819-3cbc9664e998 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:33.484121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 204/406] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:33.484121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0705d0-0a39-4eed-8131-c273b1b0e56a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:33.513984 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:33.616479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:33.630932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a889f-3cbd-4643-9819-3cbc9664e998 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:33.631450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0705d0-0a39-4eed-8131-c273b1b0e56a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:33.674357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2587921-90d1-4258-8f2f-f55f7a541a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:33.674966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2587921-90d1-4258-8f2f-f55f7a541a88 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:33.675244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2587921-90d1-4258-8f2f-f55f7a541a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:33.675569 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2587921-90d1-4258-8f2f-f55f7a541a88 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:33.676184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2587921-90d1-4258-8f2f-f55f7a541a88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:33.676486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2587921-90d1-4258-8f2f-f55f7a541a88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:33.676706 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728f4fe-fc90-47db-a811-0c0abc8dfda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:33.677872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f4fe-fc90-47db-a811-0c0abc8dfda7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:33.678147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f4fe-fc90-47db-a811-0c0abc8dfda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:33.678510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728f4fe-fc90-47db-a811-0c0abc8dfda7 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:33.678609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2587921-90d1-4258-8f2f-f55f7a541a88 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:33.679185 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728f4fe-fc90-47db-a811-0c0abc8dfda7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:33.679485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728f4fe-fc90-47db-a811-0c0abc8dfda7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:33.681997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728f4fe-fc90-47db-a811-0c0abc8dfda7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:33.695196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 205/407] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:33.696061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2587921-90d1-4258-8f2f-f55f7a541a88 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:33.702675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 203/408] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Oct 24 13:29:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:33.703770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728f4fe-fc90-47db-a811-0c0abc8dfda7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:33.789281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:33.816972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728f4fe-fc90-47db-a811-0c0abc8dfda7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:33.832202 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2587921-90d1-4258-8f2f-f55f7a541a88 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:33.859755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4590a-205d-44d6-b51b-c1b85eb003f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:33.860406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4590a-205d-44d6-b51b-c1b85eb003f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:33.860686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4590a-205d-44d6-b51b-c1b85eb003f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:33.860960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4590a-205d-44d6-b51b-c1b85eb003f8 None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/d07f8443d5594f47bc81e9f9dc9ce43b/roles/037adbca67514eda861c68218a60c50a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:33.879592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4590a-205d-44d6-b51b-c1b85eb003f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=d07f8443d5594f47bc81e9f9dc9ce43b, role_id=037adbca67514eda861c68218a60c50a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:33.881756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4590a-205d-44d6-b51b-c1b85eb003f8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:33.883394 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e2a30-ffdc-434e-ac19-19df17d9524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:33.884484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e2a30-ffdc-434e-ac19-19df17d9524f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:33.885383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e2a30-ffdc-434e-ac19-19df17d9524f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:33.885982 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e2a30-ffdc-434e-ac19-19df17d9524f None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:33.886636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e2a30-ffdc-434e-ac19-19df17d9524f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:33.887006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e2a30-ffdc-434e-ac19-19df17d9524f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:33.888517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e2a30-ffdc-434e-ac19-19df17d9524f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:33.896283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 204/409] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:29:33 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/d07f8443d5594f47bc81e9f9dc9ce43b/roles/037adbca67514eda861c68218a60c50a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:33.897369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4590a-205d-44d6-b51b-c1b85eb003f8 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:33.907176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 206/410] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:33.908680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e2a30-ffdc-434e-ac19-19df17d9524f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:33.924215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:34.017070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4590a-205d-44d6-b51b-c1b85eb003f8 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:34.073938 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:34.089105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e2a30-ffdc-434e-ac19-19df17d9524f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:34.101751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260130d7-66d3-4875-85a5-3ab986217e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:34.102463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260130d7-66d3-4875-85a5-3ab986217e27 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:34.102752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260130d7-66d3-4875-85a5-3ab986217e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:34.103032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260130d7-66d3-4875-85a5-3ab986217e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:34.122006 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260130d7-66d3-4875-85a5-3ab986217e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:34.123532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260130d7-66d3-4875-85a5-3ab986217e27 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-260130d7-66d3-4875-85a5-3ab986217e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:34.149041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:34.150115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:34.150993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:34.150993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:34.150993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:34.150993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 205/411] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:34.156912 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260130d7-66d3-4875-85a5-3ab986217e27 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:34.159087 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66374dbb-9104-46c5-bdcb-b38d432ed83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:34.160100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66374dbb-9104-46c5-bdcb-b38d432ed83a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:34.160557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66374dbb-9104-46c5-bdcb-b38d432ed83a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:34.161029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66374dbb-9104-46c5-bdcb-b38d432ed83a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:34.171177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66374dbb-9104-46c5-bdcb-b38d432ed83a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:34.172924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66374dbb-9104-46c5-bdcb-b38d432ed83a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-66374dbb-9104-46c5-bdcb-b38d432ed83a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:34.190445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:34.191368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:34.192215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:34.192215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:34.192215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:34.192215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:34.192215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:34.192215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:34.192215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:34.192215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:34.194347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 207/412] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:34.201772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66374dbb-9104-46c5-bdcb-b38d432ed83a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:34.291807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:34.343845 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260130d7-66d3-4875-85a5-3ab986217e27 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:34.362093 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66374dbb-9104-46c5-bdcb-b38d432ed83a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:34.400623 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfb0f3-4288-45a4-87f4-0413c6a78969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:34.401576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfb0f3-4288-45a4-87f4-0413c6a78969 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:34.402117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfb0f3-4288-45a4-87f4-0413c6a78969 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:34.402626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfb0f3-4288-45a4-87f4-0413c6a78969 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:34.404088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfb0f3-4288-45a4-87f4-0413c6a78969 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:34.404606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfb0f3-4288-45a4-87f4-0413c6a78969 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:34.406236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfb0f3-4288-45a4-87f4-0413c6a78969 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:34.425795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 208/413] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:34.428079 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfb0f3-4288-45a4-87f4-0413c6a78969 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:34.451448 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc4600-9b1b-47c0-96ff-6c222a8dd4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:34.452101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc4600-9b1b-47c0-96ff-6c222a8dd4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:34.452346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc4600-9b1b-47c0-96ff-6c222a8dd4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:34.452617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc4600-9b1b-47c0-96ff-6c222a8dd4f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:34.453270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc4600-9b1b-47c0-96ff-6c222a8dd4f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:34.453572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc4600-9b1b-47c0-96ff-6c222a8dd4f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:34.454968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc4600-9b1b-47c0-96ff-6c222a8dd4f7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:34.472259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 206/414] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:34.473846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc4600-9b1b-47c0-96ff-6c222a8dd4f7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:34.548489 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:34.594674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfb0f3-4288-45a4-87f4-0413c6a78969 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:34.642540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:34.655406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c7743-ce4f-4009-9a1f-fffeb2f02011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:34.656072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c7743-ce4f-4009-9a1f-fffeb2f02011 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:34.657517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c7743-ce4f-4009-9a1f-fffeb2f02011 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:34.657802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c7743-ce4f-4009-9a1f-fffeb2f02011 None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/5b8479c571ec449fbfc306d25fbf7fa2/roles/037adbca67514eda861c68218a60c50a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:34.671565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc4600-9b1b-47c0-96ff-6c222a8dd4f7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:34.686982 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c7743-ce4f-4009-9a1f-fffeb2f02011 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=5b8479c571ec449fbfc306d25fbf7fa2, role_id=037adbca67514eda861c68218a60c50a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:34.688425 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c7743-ce4f-4009-9a1f-fffeb2f02011 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:34.712226 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 209/415] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:29:34 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/5b8479c571ec449fbfc306d25fbf7fa2/roles/037adbca67514eda861c68218a60c50a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:34.725989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f22e8a-69a7-45fd-8dcf-3c1edff73d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:34.726673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f22e8a-69a7-45fd-8dcf-3c1edff73d7b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:34.726954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f22e8a-69a7-45fd-8dcf-3c1edff73d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:34.727227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f22e8a-69a7-45fd-8dcf-3c1edff73d7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:34.735231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f22e8a-69a7-45fd-8dcf-3c1edff73d7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:34.736729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f22e8a-69a7-45fd-8dcf-3c1edff73d7b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-37f22e8a-69a7-45fd-8dcf-3c1edff73d7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:34.750296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:34.751213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:34.752302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:34.752302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:34.752302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:34.752302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:34.752302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:34.752302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:34.752302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:34.752302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:34.752302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 207/416] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:34.759406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c7743-ce4f-4009-9a1f-fffeb2f02011 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:34.924066 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c7743-ce4f-4009-9a1f-fffeb2f02011 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:34.979896 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22026ae-85c0-490f-a884-ef44b867b73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:34.980512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22026ae-85c0-490f-a884-ef44b867b73a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:34.980792 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22026ae-85c0-490f-a884-ef44b867b73a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:34.981071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22026ae-85c0-490f-a884-ef44b867b73a None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:34.981764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22026ae-85c0-490f-a884-ef44b867b73a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:34.982072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22026ae-85c0-490f-a884-ef44b867b73a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:34.983541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22026ae-85c0-490f-a884-ef44b867b73a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:35.001413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 210/417] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:35.008606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f22e8a-69a7-45fd-8dcf-3c1edff73d7b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:35.122738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:35.129060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f22e8a-69a7-45fd-8dcf-3c1edff73d7b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:35.166596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70caf4-3d86-4968-8800-1f98cf2e1970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:35.167250 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70caf4-3d86-4968-8800-1f98cf2e1970 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:35.167523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70caf4-3d86-4968-8800-1f98cf2e1970 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:35.167790 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70caf4-3d86-4968-8800-1f98cf2e1970 None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/3c8c594699c045a09c835910d894896b/roles/037adbca67514eda861c68218a60c50a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:35.182409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70caf4-3d86-4968-8800-1f98cf2e1970 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=3c8c594699c045a09c835910d894896b, role_id=037adbca67514eda861c68218a60c50a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:35.183769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70caf4-3d86-4968-8800-1f98cf2e1970 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:35.210430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 208/418] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:29:35 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/3c8c594699c045a09c835910d894896b/roles/037adbca67514eda861c68218a60c50a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:35.623808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e40e2d-a98c-45e5-99df-bedb46c1e2b2 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:35.624538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e40e2d-a98c-45e5-99df-bedb46c1e2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:35.625025 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e40e2d-a98c-45e5-99df-bedb46c1e2b2 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:35.626890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 211/419] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:35.633223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e4bd-3424-4cf3-958c-2f77a3a45184 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:35.634077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e4bd-3424-4cf3-958c-2f77a3a45184 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:35.634678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51e4bd-3424-4cf3-958c-2f77a3a45184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:35.653866 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-de51e4bd-3424-4cf3-958c-2f77a3a45184 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:35.655390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-de51e4bd-3424-4cf3-958c-2f77a3a45184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:35.723549 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 209/420] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:36.082301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd65a0a-5db3-42c6-a092-699b6fe256e5 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:36.082842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd65a0a-5db3-42c6-a092-699b6fe256e5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:36.082842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd65a0a-5db3-42c6-a092-699b6fe256e5 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:36.084515 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 212/421] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:36.089451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2314343-278a-43a9-92c2-5cc0db1f12a0 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:36.089967 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2314343-278a-43a9-92c2-5cc0db1f12a0 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:36.090483 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2314343-278a-43a9-92c2-5cc0db1f12a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:36.115194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f2314343-278a-43a9-92c2-5cc0db1f12a0 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:36.116778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f2314343-278a-43a9-92c2-5cc0db1f12a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:36.142537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd65a0a-5db3-42c6-a092-699b6fe256e5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:36.190026 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 210/422] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:36.233593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:36.325948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd65a0a-5db3-42c6-a092-699b6fe256e5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:36.368255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d065d-3977-4721-a3cc-159707aaa347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:36.368812 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d065d-3977-4721-a3cc-159707aaa347 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:36.369088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d065d-3977-4721-a3cc-159707aaa347 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:36.369353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d065d-3977-4721-a3cc-159707aaa347 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:36.383294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d065d-3977-4721-a3cc-159707aaa347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-a28d065d-3977-4721-a3cc-159707aaa347 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:36.388407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:36.389285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:36.390086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:36.394208 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 213/423] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:36.395424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2314343-278a-43a9-92c2-5cc0db1f12a0 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:36.406819 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:36.465479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d065d-3977-4721-a3cc-159707aaa347 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:36.568929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2314343-278a-43a9-92c2-5cc0db1f12a0 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:36.617227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec544e04-9a39-4e43-90cb-65ec1e4c0433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:36.618230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec544e04-9a39-4e43-90cb-65ec1e4c0433 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:36.619129 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:36.619327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec544e04-9a39-4e43-90cb-65ec1e4c0433 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:36.620916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec544e04-9a39-4e43-90cb-65ec1e4c0433 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:36.621976 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec544e04-9a39-4e43-90cb-65ec1e4c0433 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:36.622523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec544e04-9a39-4e43-90cb-65ec1e4c0433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:36.623074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d065d-3977-4721-a3cc-159707aaa347 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:36.624386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec544e04-9a39-4e43-90cb-65ec1e4c0433 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:36.635415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 211/424] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:36.637354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72324d-3bbe-4cdf-90d8-ef21ab806f1a None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:36.637809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72324d-3bbe-4cdf-90d8-ef21ab806f1a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:36.638263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72324d-3bbe-4cdf-90d8-ef21ab806f1a None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:36.640091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 212/425] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:36.644512 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72324d-3bbe-4cdf-90d8-ef21ab806f1a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:36.672013 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c01d9-5220-48e4-b32a-21a5141710b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:36.672643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c01d9-5220-48e4-b32a-21a5141710b0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:36.672915 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c01d9-5220-48e4-b32a-21a5141710b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:36.673178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c01d9-5220-48e4-b32a-21a5141710b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:36.687658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c01d9-5220-48e4-b32a-21a5141710b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-e66c01d9-5220-48e4-b32a-21a5141710b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:36.693878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:36.694883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:36.695661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 214/426] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:36.696782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ae44f-3849-45c6-8a4e-c630fc8d262a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:36.696952 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ae44f-3849-45c6-8a4e-c630fc8d262a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:36.697586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ae44f-3849-45c6-8a4e-c630fc8d262a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:36.720808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-215ae44f-3849-45c6-8a4e-c630fc8d262a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:36.722011 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-215ae44f-3849-45c6-8a4e-c630fc8d262a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:36.723923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:36.770240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72324d-3bbe-4cdf-90d8-ef21ab806f1a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:36.810353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 215/427] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:36.810353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ae44f-3849-45c6-8a4e-c630fc8d262a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:36.814202 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4844d47-ff9c-4d0e-8aab-c39466458d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:36.814968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4844d47-ff9c-4d0e-8aab-c39466458d63 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:36.814968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4844d47-ff9c-4d0e-8aab-c39466458d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:36.815160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4844d47-ff9c-4d0e-8aab-c39466458d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:36.839945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4844d47-ff9c-4d0e-8aab-c39466458d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:36.841397 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4844d47-ff9c-4d0e-8aab-c39466458d63 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-d4844d47-ff9c-4d0e-8aab-c39466458d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:36.871628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:36.872536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:36.873387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:36.873387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:36.873387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:36.874744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 213/428] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:36.883090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4844d47-ff9c-4d0e-8aab-c39466458d63 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:37.006864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:37.013172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4844d47-ff9c-4d0e-8aab-c39466458d63 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:37.038534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:37.048273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ae44f-3849-45c6-8a4e-c630fc8d262a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:37.048433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d4d23-1562-42f6-9ce8-aa00ee9fe654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:37.048967 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d4d23-1562-42f6-9ce8-aa00ee9fe654 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.049248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d4d23-1562-42f6-9ce8-aa00ee9fe654 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.049518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d4d23-1562-42f6-9ce8-aa00ee9fe654 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.050369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d4d23-1562-42f6-9ce8-aa00ee9fe654 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:37.050670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d4d23-1562-42f6-9ce8-aa00ee9fe654 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:37.053648 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d4d23-1562-42f6-9ce8-aa00ee9fe654 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:37.066490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 214/429] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:37.073632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d4d23-1562-42f6-9ce8-aa00ee9fe654 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:37.131072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42776d70-f772-47a7-9d37-2815cd9f2508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:37.132464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42776d70-f772-47a7-9d37-2815cd9f2508 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.133015 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42776d70-f772-47a7-9d37-2815cd9f2508 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.133504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42776d70-f772-47a7-9d37-2815cd9f2508 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.134395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42776d70-f772-47a7-9d37-2815cd9f2508 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:37.134996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42776d70-f772-47a7-9d37-2815cd9f2508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:37.136659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42776d70-f772-47a7-9d37-2815cd9f2508 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:37.145817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 216/430] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:37.147096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42776d70-f772-47a7-9d37-2815cd9f2508 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:37.180496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d4d23-1562-42f6-9ce8-aa00ee9fe654 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:37.216630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b7c0a-d4f6-471f-8efb-314347ff44b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:37.217414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b7c0a-d4f6-471f-8efb-314347ff44b1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.217695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b7c0a-d4f6-471f-8efb-314347ff44b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.217962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b7c0a-d4f6-471f-8efb-314347ff44b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.226322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b7c0a-d4f6-471f-8efb-314347ff44b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:37.227931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b7c0a-d4f6-471f-8efb-314347ff44b1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-788b7c0a-d4f6-471f-8efb-314347ff44b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:37.242458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:37.243373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:37.244255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:37.244255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:37.244255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:37.244255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:37.244255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:37.244255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:37.244255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:37.244255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:37.244255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:29:37.244255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:37.248240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 215/431] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Oct 24 13:29:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:37.249059 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b7c0a-d4f6-471f-8efb-314347ff44b1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:37.265984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42776d70-f772-47a7-9d37-2815cd9f2508 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:37.297349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:37.307234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5b0f6-2a02-4036-ba0d-b48dff911dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:37.309560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5b0f6-2a02-4036-ba0d-b48dff911dcb None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.310103 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5b0f6-2a02-4036-ba0d-b48dff911dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.310629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e5b0f6-2a02-4036-ba0d-b48dff911dcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.322801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e5b0f6-2a02-4036-ba0d-b48dff911dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-19e5b0f6-2a02-4036-ba0d-b48dff911dcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:37.330149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:37.331040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:37.331840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:37.338257 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 217/432] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:37.339402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5b0f6-2a02-4036-ba0d-b48dff911dcb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:37.354904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:37.363380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b7c0a-d4f6-471f-8efb-314347ff44b1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:37.403383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3a4b3-8d51-4d3a-b660-1b3a2b219c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:37.403928 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3a4b3-8d51-4d3a-b660-1b3a2b219c34 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.404202 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3a4b3-8d51-4d3a-b660-1b3a2b219c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.404467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3a4b3-8d51-4d3a-b660-1b3a2b219c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.416919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3a4b3-8d51-4d3a-b660-1b3a2b219c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:37.419025 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3a4b3-8d51-4d3a-b660-1b3a2b219c34 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6cf3a4b3-8d51-4d3a-b660-1b3a2b219c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:37.433615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:37.434623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:37.435486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:37.435486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:37.435486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:37.435486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 216/433] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:37.436285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3a4b3-8d51-4d3a-b660-1b3a2b219c34 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:37.462935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e5b0f6-2a02-4036-ba0d-b48dff911dcb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:37.502991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccbc93-0e57-44c4-832f-60170604c34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:37.503959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccbc93-0e57-44c4-832f-60170604c34e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.504550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccbc93-0e57-44c4-832f-60170604c34e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.505041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccbc93-0e57-44c4-832f-60170604c34e None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.506593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccbc93-0e57-44c4-832f-60170604c34e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:37.507100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccbc93-0e57-44c4-832f-60170604c34e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:37.509273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccbc93-0e57-44c4-832f-60170604c34e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:37.529130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 218/434] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Oct 24 13:29:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:37.530384 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccbc93-0e57-44c4-832f-60170604c34e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:37.545770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:37.555876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3a4b3-8d51-4d3a-b660-1b3a2b219c34 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:37.592911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfcf67-153c-4b80-bad9-1a4aa1c417d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:37.593521 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfcf67-153c-4b80-bad9-1a4aa1c417d8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.593791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfcf67-153c-4b80-bad9-1a4aa1c417d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.594057 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfcf67-153c-4b80-bad9-1a4aa1c417d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.594732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfcf67-153c-4b80-bad9-1a4aa1c417d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:37.595028 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfcf67-153c-4b80-bad9-1a4aa1c417d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:37.596392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfcf67-153c-4b80-bad9-1a4aa1c417d8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:37.606791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 217/435] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:37.607683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfcf67-153c-4b80-bad9-1a4aa1c417d8 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:37.612855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:37.649340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccbc93-0e57-44c4-832f-60170604c34e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:37.685121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdb1d7-caa5-4ada-8a9c-09c3355835a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:37.685646 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdb1d7-caa5-4ada-8a9c-09c3355835a2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.685919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdb1d7-caa5-4ada-8a9c-09c3355835a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.686206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdb1d7-caa5-4ada-8a9c-09c3355835a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.686844 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdb1d7-caa5-4ada-8a9c-09c3355835a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:37.687140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdb1d7-caa5-4ada-8a9c-09c3355835a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:37.688496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdb1d7-caa5-4ada-8a9c-09c3355835a2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:37.705091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 219/436] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:37.705091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7688e9-666f-49e8-a424-180c52270711 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.705091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7688e9-666f-49e8-a424-180c52270711 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.705091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7688e9-666f-49e8-a424-180c52270711 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.705091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 220/437] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:37.705091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7688e9-666f-49e8-a424-180c52270711 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:37.725639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfcf67-153c-4b80-bad9-1a4aa1c417d8 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:37.760402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073c809-da8e-46e2-b06d-ff7caee3d458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:37.761007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073c809-da8e-46e2-b06d-ff7caee3d458 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.761284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073c809-da8e-46e2-b06d-ff7caee3d458 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.761553 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073c809-da8e-46e2-b06d-ff7caee3d458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.785671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:37.814555 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7688e9-666f-49e8-a424-180c52270711 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:37.840131 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073c809-da8e-46e2-b06d-ff7caee3d458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:37.841600 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073c809-da8e-46e2-b06d-ff7caee3d458 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:37.845040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:37.855129 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e074912-6bfe-4882-b112-3a918b503757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:37.855785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e074912-6bfe-4882-b112-3a918b503757 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.856091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e074912-6bfe-4882-b112-3a918b503757 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.856362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e074912-6bfe-4882-b112-3a918b503757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.868654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e074912-6bfe-4882-b112-3a918b503757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:37.870136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e074912-6bfe-4882-b112-3a918b503757 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-7e074912-6bfe-4882-b112-3a918b503757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:37.885877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:37.886898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:37.887719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:37.887719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:37.887719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:37.890497 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 221/438] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:37.892097 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e074912-6bfe-4882-b112-3a918b503757 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:37.956163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 218/439] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:29:37.958286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605751fd-2b66-4a03-be82-1981c32c48c4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:37.959131 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605751fd-2b66-4a03-be82-1981c32c48c4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:37.959626 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605751fd-2b66-4a03-be82-1981c32c48c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:37.980305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-605751fd-2b66-4a03-be82-1981c32c48c4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:37.981476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-605751fd-2b66-4a03-be82-1981c32c48c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:37.997596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e074912-6bfe-4882-b112-3a918b503757 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:38.029265 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:38.030408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d4b11-94ab-4c33-a99b-b643bcb43825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:38.031077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d4b11-94ab-4c33-a99b-b643bcb43825 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.031330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d4b11-94ab-4c33-a99b-b643bcb43825 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.031654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d4b11-94ab-4c33-a99b-b643bcb43825 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.040224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d4b11-94ab-4c33-a99b-b643bcb43825 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:38.041919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d4b11-94ab-4c33-a99b-b643bcb43825 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:38.047507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 219/440] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:38.048393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605751fd-2b66-4a03-be82-1981c32c48c4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-006d4b11-94ab-4c33-a99b-b643bcb43825 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:38.055035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:38.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 222/441] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:29:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:38.061853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d4b11-94ab-4c33-a99b-b643bcb43825 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.104943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:38.157598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605751fd-2b66-4a03-be82-1981c32c48c4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:38.165482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d4b11-94ab-4c33-a99b-b643bcb43825 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:38.195452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecce204-4177-4fdc-933c-acde34737266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:38.198821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecce204-4177-4fdc-933c-acde34737266 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.198821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecce204-4177-4fdc-933c-acde34737266 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.198821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecce204-4177-4fdc-933c-acde34737266 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.198821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecce204-4177-4fdc-933c-acde34737266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:38.198821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecce204-4177-4fdc-933c-acde34737266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:38.199168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecce204-4177-4fdc-933c-acde34737266 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:38.203765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e0a09-4bd2-4319-9462-c34dd9692f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:38.204404 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e0a09-4bd2-4319-9462-c34dd9692f5d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.204686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e0a09-4bd2-4319-9462-c34dd9692f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.207783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e0a09-4bd2-4319-9462-c34dd9692f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.216835 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e0a09-4bd2-4319-9462-c34dd9692f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:38.218390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e0a09-4bd2-4319-9462-c34dd9692f5d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:38.220176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 220/442] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:38.221606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7d742-d465-4acc-80e4-14903d194476 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.221885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7d742-d465-4acc-80e4-14903d194476 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.222152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7d742-d465-4acc-80e4-14903d194476 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.226752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 221/443] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:38.226752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7d742-d465-4acc-80e4-14903d194476 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-4f0e0a09-4bd2-4319-9462-c34dd9692f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:38.234565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:38.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:38.236600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:38.236600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:38.236600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:38.236600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 223/444] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:38.236600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e0a09-4bd2-4319-9462-c34dd9692f5d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.279397 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:38.333847 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:38.338530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7d742-d465-4acc-80e4-14903d194476 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:38.343383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e0a09-4bd2-4319-9462-c34dd9692f5d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:38.371569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b5c7e-62d2-4282-b6ce-a4329ca2d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:38.372167 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b5c7e-62d2-4282-b6ce-a4329ca2d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.372442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b5c7e-62d2-4282-b6ce-a4329ca2d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.372707 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b5c7e-62d2-4282-b6ce-a4329ca2d9d0 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.373413 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b5c7e-62d2-4282-b6ce-a4329ca2d9d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:38.373707 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b5c7e-62d2-4282-b6ce-a4329ca2d9d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:38.375157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b5c7e-62d2-4282-b6ce-a4329ca2d9d0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:38.376568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e41c4d-fb88-4cd0-95df-c638c2599936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:38.377190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e41c4d-fb88-4cd0-95df-c638c2599936 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.377525 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e41c4d-fb88-4cd0-95df-c638c2599936 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.377797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e41c4d-fb88-4cd0-95df-c638c2599936 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.385367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e41c4d-fb88-4cd0-95df-c638c2599936 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:38.386835 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e41c4d-fb88-4cd0-95df-c638c2599936 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:38.391888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 222/445] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:38.393059 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdcff2-ef0d-433d-a9ac-1288645cb737 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.393341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdcff2-ef0d-433d-a9ac-1288645cb737 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.393612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdcff2-ef0d-433d-a9ac-1288645cb737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-d6e41c4d-fb88-4cd0-95df-c638c2599936 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:38.398516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:38.399396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:38.400268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:38.400268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:38.400268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:38.400268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:38.400268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:38.400268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:38.400268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:38.400268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:38.400268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:29:38.400268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:38.400268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 224/446] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:38.400815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e41c4d-fb88-4cd0-95df-c638c2599936 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.416669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-aecdcff2-ef0d-433d-a9ac-1288645cb737 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:38.417851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-aecdcff2-ef0d-433d-a9ac-1288645cb737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:38.483950 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 223/447] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:38.484695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdcff2-ef0d-433d-a9ac-1288645cb737 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.504798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e41c4d-fb88-4cd0-95df-c638c2599936 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:38.516954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:38.544180 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d22d70-6fee-41ce-b1f9-1442cbac131c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:38.544786 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d22d70-6fee-41ce-b1f9-1442cbac131c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.545063 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d22d70-6fee-41ce-b1f9-1442cbac131c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.545330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d22d70-6fee-41ce-b1f9-1442cbac131c None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.545959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d22d70-6fee-41ce-b1f9-1442cbac131c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:38.546284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d22d70-6fee-41ce-b1f9-1442cbac131c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:38.547634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d22d70-6fee-41ce-b1f9-1442cbac131c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:38.564178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 225/448] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:38.564178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d22d70-6fee-41ce-b1f9-1442cbac131c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.575362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:38.603440 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdcff2-ef0d-433d-a9ac-1288645cb737 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:38.641466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ed038-1be2-4327-bd78-217ea102034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:38.642094 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ed038-1be2-4327-bd78-217ea102034d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.642394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ed038-1be2-4327-bd78-217ea102034d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.642661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ed038-1be2-4327-bd78-217ea102034d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.654137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ed038-1be2-4327-bd78-217ea102034d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-978ed038-1be2-4327-bd78-217ea102034d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:38.658854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:38.659711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:38.660513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 224/449] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:38.661193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ed038-1be2-4327-bd78-217ea102034d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.672255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d22d70-6fee-41ce-b1f9-1442cbac131c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:38.709973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa742e1-b7e9-4808-8819-7bd95cf7c2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:38.710637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa742e1-b7e9-4808-8819-7bd95cf7c2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.710920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa742e1-b7e9-4808-8819-7bd95cf7c2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.711193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa742e1-b7e9-4808-8819-7bd95cf7c2b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.721798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa742e1-b7e9-4808-8819-7bd95cf7c2b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:38.723311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa742e1-b7e9-4808-8819-7bd95cf7c2b2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-cfa742e1-b7e9-4808-8819-7bd95cf7c2b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:38.738766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:38.740350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:38.741484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:38.741484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:38.741484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:38.741484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 226/450] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:38.741484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa742e1-b7e9-4808-8819-7bd95cf7c2b2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.763855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:38.772813 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ed038-1be2-4327-bd78-217ea102034d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:38.807227 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:38.808363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008d32d-7514-48e8-8983-c8a22014287f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:38.809064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008d32d-7514-48e8-8983-c8a22014287f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.809344 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008d32d-7514-48e8-8983-c8a22014287f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.809612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008d32d-7514-48e8-8983-c8a22014287f None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.810383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008d32d-7514-48e8-8983-c8a22014287f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:38.810682 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008d32d-7514-48e8-8983-c8a22014287f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:38.812106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008d32d-7514-48e8-8983-c8a22014287f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:38.829620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 225/451] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:38.830492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008d32d-7514-48e8-8983-c8a22014287f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.850180 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa742e1-b7e9-4808-8819-7bd95cf7c2b2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:38.883750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899b386-f6db-45fa-ae93-ea7f150394f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:38.884375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899b386-f6db-45fa-ae93-ea7f150394f7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.884647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899b386-f6db-45fa-ae93-ea7f150394f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.884917 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899b386-f6db-45fa-ae93-ea7f150394f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.891341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899b386-f6db-45fa-ae93-ea7f150394f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:38.892753 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899b386-f6db-45fa-ae93-ea7f150394f7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b899b386-f6db-45fa-ae93-ea7f150394f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:38.904027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:38.904949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:38.905794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:38.905794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:38.905794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:38.905794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:38.905794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:38.905794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:38.905794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:38.905794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:38.905794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 227/452] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:38.906278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899b386-f6db-45fa-ae93-ea7f150394f7 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.935241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008d32d-7514-48e8-8983-c8a22014287f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:38.969186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74b350-cd7b-4b2b-ad8b-03b90728c8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:38.969788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b350-cd7b-4b2b-ad8b-03b90728c8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:38.970066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b350-cd7b-4b2b-ad8b-03b90728c8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:38.970365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b350-cd7b-4b2b-ad8b-03b90728c8e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:38.971018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74b350-cd7b-4b2b-ad8b-03b90728c8e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:38.971320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74b350-cd7b-4b2b-ad8b-03b90728c8e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:38.972705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74b350-cd7b-4b2b-ad8b-03b90728c8e7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:38.981327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 226/453] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:38.982217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74b350-cd7b-4b2b-ad8b-03b90728c8e7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:38.998514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:39.010036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899b386-f6db-45fa-ae93-ea7f150394f7 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.033168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:39.045048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22643e8d-a912-4fa9-b4e0-8951d157abb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:39.045653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22643e8d-a912-4fa9-b4e0-8951d157abb9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:39.045932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22643e8d-a912-4fa9-b4e0-8951d157abb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:39.046228 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22643e8d-a912-4fa9-b4e0-8951d157abb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:39.046872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22643e8d-a912-4fa9-b4e0-8951d157abb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:39.047198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22643e8d-a912-4fa9-b4e0-8951d157abb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:39.048555 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22643e8d-a912-4fa9-b4e0-8951d157abb9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:39.060457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 228/454] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:39.061255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22643e8d-a912-4fa9-b4e0-8951d157abb9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:39.085534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74b350-cd7b-4b2b-ad8b-03b90728c8e7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.119099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a5063-66c3-4b81-85da-8cdad49ff80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:39.119718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5063-66c3-4b81-85da-8cdad49ff80f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:39.119989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5063-66c3-4b81-85da-8cdad49ff80f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:39.120251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a5063-66c3-4b81-85da-8cdad49ff80f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:39.131403 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a5063-66c3-4b81-85da-8cdad49ff80f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-440a5063-66c3-4b81-85da-8cdad49ff80f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:39.136582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:39.137657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:39.138695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 227/455] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:39.139512 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a5063-66c3-4b81-85da-8cdad49ff80f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:39.163942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22643e8d-a912-4fa9-b4e0-8951d157abb9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.198374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822355b-fa39-446f-9ab0-a55826b4b304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:39.199016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822355b-fa39-446f-9ab0-a55826b4b304 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:39.199242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822355b-fa39-446f-9ab0-a55826b4b304 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:39.199511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822355b-fa39-446f-9ab0-a55826b4b304 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:39.213600 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:39.245480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a5063-66c3-4b81-85da-8cdad49ff80f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.252271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:39.268056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822355b-fa39-446f-9ab0-a55826b4b304 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:39.269528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822355b-fa39-446f-9ab0-a55826b4b304 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:39.279989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41bf50-0df2-40f9-a7e3-009e76d27af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:39.280586 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41bf50-0df2-40f9-a7e3-009e76d27af7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:39.280861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41bf50-0df2-40f9-a7e3-009e76d27af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:39.281126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41bf50-0df2-40f9-a7e3-009e76d27af7 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:39.281802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41bf50-0df2-40f9-a7e3-009e76d27af7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:39.282098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41bf50-0df2-40f9-a7e3-009e76d27af7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:39.283532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41bf50-0df2-40f9-a7e3-009e76d27af7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:39.301835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 228/456] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:39.302735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41bf50-0df2-40f9-a7e3-009e76d27af7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:39.376621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 229/457] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:29:39.377447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822355b-fa39-446f-9ab0-a55826b4b304 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:39.404753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41bf50-0df2-40f9-a7e3-009e76d27af7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.434614 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:39.440076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb944e-3f4a-44d2-8d60-e1c1c36d1b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:39.440703 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb944e-3f4a-44d2-8d60-e1c1c36d1b5b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:39.440980 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb944e-3f4a-44d2-8d60-e1c1c36d1b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:39.441247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb944e-3f4a-44d2-8d60-e1c1c36d1b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:39.453016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb944e-3f4a-44d2-8d60-e1c1c36d1b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:39.454847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb944e-3f4a-44d2-8d60-e1c1c36d1b5b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:39.462027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-05eb944e-3f4a-44d2-8d60-e1c1c36d1b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:39.470425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:39.471335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:39.472168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:39.472168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:39.472168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:39.472168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 229/458] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:39.472439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb944e-3f4a-44d2-8d60-e1c1c36d1b5b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:39.480265 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822355b-fa39-446f-9ab0-a55826b4b304 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.516235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33293fca-0559-4372-9983-9c7a7566bf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:39.516783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293fca-0559-4372-9983-9c7a7566bf1b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:39.517060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293fca-0559-4372-9983-9c7a7566bf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:39.517327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293fca-0559-4372-9983-9c7a7566bf1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:39.524586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33293fca-0559-4372-9983-9c7a7566bf1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:39.526022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33293fca-0559-4372-9983-9c7a7566bf1b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-33293fca-0559-4372-9983-9c7a7566bf1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:39.537024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:39.538071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:39.538963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:39.538963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:39.538963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:39.538963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:39.538963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:39.538963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:39.538963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:39.538963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:39.538963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 230/459] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:39.538963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33293fca-0559-4372-9983-9c7a7566bf1b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:39.574971 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb944e-3f4a-44d2-8d60-e1c1c36d1b5b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.608067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6456b-1ec2-45de-ad95-6ca476bd695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:39.608665 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6456b-1ec2-45de-ad95-6ca476bd695e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:39.608941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6456b-1ec2-45de-ad95-6ca476bd695e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:39.609209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6456b-1ec2-45de-ad95-6ca476bd695e None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:39.609856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6456b-1ec2-45de-ad95-6ca476bd695e None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:39.610175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6456b-1ec2-45de-ad95-6ca476bd695e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:39.611534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6456b-1ec2-45de-ad95-6ca476bd695e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:39.619199 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 230/460] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:39.619972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6456b-1ec2-45de-ad95-6ca476bd695e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:39.640332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33293fca-0559-4372-9983-9c7a7566bf1b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.670428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:39.675452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd324eb-9676-4b49-aca9-eb6e7a0cfc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:39.676071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd324eb-9676-4b49-aca9-eb6e7a0cfc01 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:39.676343 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd324eb-9676-4b49-aca9-eb6e7a0cfc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:39.676608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd324eb-9676-4b49-aca9-eb6e7a0cfc01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:39.678077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd324eb-9676-4b49-aca9-eb6e7a0cfc01 None admin] RBAC: Adding query filter params (role.id=037adbca67514eda861c68218a60c50a, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=18bb93669846435a85eb34771cfb09fe) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:39.678526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd324eb-9676-4b49-aca9-eb6e7a0cfc01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:39.679945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd324eb-9676-4b49-aca9-eb6e7a0cfc01 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:39.685514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 231/461] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:29:39 2022] GET /identity/v3/role_assignments?user.id=18bb93669846435a85eb34771cfb09fe&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=037adbca67514eda861c68218a60c50a => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:39.686304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd324eb-9676-4b49-aca9-eb6e7a0cfc01 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:39.705355 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:39.744280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6456b-1ec2-45de-ad95-6ca476bd695e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.783085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b326fb-9b32-41ca-891f-f9150b34d383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:39.783738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b326fb-9b32-41ca-891f-f9150b34d383 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:39.783953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b326fb-9b32-41ca-891f-f9150b34d383 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:39.784164 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b326fb-9b32-41ca-891f-f9150b34d383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:39.797113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b326fb-9b32-41ca-891f-f9150b34d383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:39.798594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b326fb-9b32-41ca-891f-f9150b34d383 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:39.808169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd324eb-9676-4b49-aca9-eb6e7a0cfc01 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-c2b326fb-9b32-41ca-891f-f9150b34d383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:39.814234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:39.815282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:39.816178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:39.816178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:39.816178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:39.818527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 231/462] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:39.819781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b326fb-9b32-41ca-891f-f9150b34d383 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:39.852819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5bf3a-756d-4ab1-8e69-73201639a95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:39.853538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5bf3a-756d-4ab1-8e69-73201639a95f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:39.853989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5bf3a-756d-4ab1-8e69-73201639a95f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:39.854237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5bf3a-756d-4ab1-8e69-73201639a95f None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:39.854951 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5bf3a-756d-4ab1-8e69-73201639a95f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:39.855319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5bf3a-756d-4ab1-8e69-73201639a95f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:39.856747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5bf3a-756d-4ab1-8e69-73201639a95f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:39.869234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 232/463] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:39.870145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5bf3a-756d-4ab1-8e69-73201639a95f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:39.933673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:39.938058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b326fb-9b32-41ca-891f-f9150b34d383 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.945025 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:39.976566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378fd78-702e-4f78-8866-96f0de702f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:39.977180 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378fd78-702e-4f78-8866-96f0de702f01 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:39.977459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378fd78-702e-4f78-8866-96f0de702f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:39.977736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378fd78-702e-4f78-8866-96f0de702f01 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:39.978451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378fd78-702e-4f78-8866-96f0de702f01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:39.978751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378fd78-702e-4f78-8866-96f0de702f01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:39.980171 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378fd78-702e-4f78-8866-96f0de702f01 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:39.983581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5bf3a-756d-4ab1-8e69-73201639a95f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:39.993725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 232/464] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:39.994590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378fd78-702e-4f78-8866-96f0de702f01 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:40.026134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e62d4-01c8-452a-8330-ed63e19a9e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:40.026795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e62d4-01c8-452a-8330-ed63e19a9e2b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:40.027076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e62d4-01c8-452a-8330-ed63e19a9e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:40.027553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e62d4-01c8-452a-8330-ed63e19a9e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:40.037756 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e62d4-01c8-452a-8330-ed63e19a9e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:40.039195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e62d4-01c8-452a-8330-ed63e19a9e2b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-699e62d4-01c8-452a-8330-ed63e19a9e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:40.054736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:40.055671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:40.056544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:40.056544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:40.056544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:40.056544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 233/465] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:40.056759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e62d4-01c8-452a-8330-ed63e19a9e2b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:40.105838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378fd78-702e-4f78-8866-96f0de702f01 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:40.145211 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63c6bf-d368-459f-b173-e79069c3d29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:40.145823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63c6bf-d368-459f-b173-e79069c3d29e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:40.146103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63c6bf-d368-459f-b173-e79069c3d29e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:40.146403 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63c6bf-d368-459f-b173-e79069c3d29e None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:40.147039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63c6bf-d368-459f-b173-e79069c3d29e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:40.147337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63c6bf-d368-459f-b173-e79069c3d29e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:40.148726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63c6bf-d368-459f-b173-e79069c3d29e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:40.162083 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e62d4-01c8-452a-8330-ed63e19a9e2b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:40.162257 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 233/466] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:40.163016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63c6bf-d368-459f-b173-e79069c3d29e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:40.185949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:40.186351 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:40.205490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2272c1-9471-4e3f-b20a-7c864ca6d27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:40.206756 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2272c1-9471-4e3f-b20a-7c864ca6d27d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:40.207038 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2272c1-9471-4e3f-b20a-7c864ca6d27d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:40.207315 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2272c1-9471-4e3f-b20a-7c864ca6d27d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:40.216993 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2272c1-9471-4e3f-b20a-7c864ca6d27d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:40.218541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2272c1-9471-4e3f-b20a-7c864ca6d27d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-2c2272c1-9471-4e3f-b20a-7c864ca6d27d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:40.229885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:40.230980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 234/467] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:40.231857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2272c1-9471-4e3f-b20a-7c864ca6d27d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:40.288072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63c6bf-d368-459f-b173-e79069c3d29e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:40.324780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624867bd-cff1-42a3-b6f5-e5623b880cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:40.325439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624867bd-cff1-42a3-b6f5-e5623b880cfe None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:40.325721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624867bd-cff1-42a3-b6f5-e5623b880cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:40.325991 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624867bd-cff1-42a3-b6f5-e5623b880cfe None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/0ea0d300a51c41fba4d4193dd2251a50/roles/037adbca67514eda861c68218a60c50a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:40.336929 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2272c1-9471-4e3f-b20a-7c864ca6d27d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:40.340832 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624867bd-cff1-42a3-b6f5-e5623b880cfe None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=0ea0d300a51c41fba4d4193dd2251a50, role_id=037adbca67514eda861c68218a60c50a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:40.342300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624867bd-cff1-42a3-b6f5-e5623b880cfe None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:40.355724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 234/468] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:29:40 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/0ea0d300a51c41fba4d4193dd2251a50/roles/037adbca67514eda861c68218a60c50a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:40.356547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624867bd-cff1-42a3-b6f5-e5623b880cfe None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:40.393730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff65689-e64e-4e60-a280-e568f1315ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:40.395069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff65689-e64e-4e60-a280-e568f1315ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:40.396583 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff65689-e64e-4e60-a280-e568f1315ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:40.397163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff65689-e64e-4e60-a280-e568f1315ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:40.398054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff65689-e64e-4e60-a280-e568f1315ee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:40.398610 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff65689-e64e-4e60-a280-e568f1315ee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:40.400239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff65689-e64e-4e60-a280-e568f1315ee9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:40.420845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 235/469] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:40.420845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff65689-e64e-4e60-a280-e568f1315ee9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:40.441686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:40.484506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:40.488622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624867bd-cff1-42a3-b6f5-e5623b880cfe None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:40.527455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f85fc-b5b4-4cce-8dac-8b32e71b3e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:40.528501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f85fc-b5b4-4cce-8dac-8b32e71b3e09 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:40.529008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f85fc-b5b4-4cce-8dac-8b32e71b3e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:40.529473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f85fc-b5b4-4cce-8dac-8b32e71b3e09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:40.538882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f85fc-b5b4-4cce-8dac-8b32e71b3e09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:40.540949 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f85fc-b5b4-4cce-8dac-8b32e71b3e09 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-002f85fc-b5b4-4cce-8dac-8b32e71b3e09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:40.553567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:40.554547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:40.560317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:40.560317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:40.560317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:40.560317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:40.560317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:40.560317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:40.560317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:40.560317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:40.560317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 235/470] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:40.560317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f85fc-b5b4-4cce-8dac-8b32e71b3e09 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:40.567228 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff65689-e64e-4e60-a280-e568f1315ee9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:40.609861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb430e-4019-488b-9fb7-8462a726ac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:40.610504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb430e-4019-488b-9fb7-8462a726ac68 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:40.610780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb430e-4019-488b-9fb7-8462a726ac68 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:40.611050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb430e-4019-488b-9fb7-8462a726ac68 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:40.612168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb430e-4019-488b-9fb7-8462a726ac68 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:40.612472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb430e-4019-488b-9fb7-8462a726ac68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:40.615172 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb430e-4019-488b-9fb7-8462a726ac68 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:40.641327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 236/471] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Oct 24 13:29:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:40.642503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb430e-4019-488b-9fb7-8462a726ac68 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:40.680120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f85fc-b5b4-4cce-8dac-8b32e71b3e09 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:40.708003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:40.717371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea8ce3-d25a-4a2b-9ff5-edc1982c6ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:40.718486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea8ce3-d25a-4a2b-9ff5-edc1982c6ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:40.718978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea8ce3-d25a-4a2b-9ff5-edc1982c6ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:40.719441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea8ce3-d25a-4a2b-9ff5-edc1982c6ea1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:40.728165 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea8ce3-d25a-4a2b-9ff5-edc1982c6ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:40.730054 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea8ce3-d25a-4a2b-9ff5-edc1982c6ea1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e2ea8ce3-d25a-4a2b-9ff5-edc1982c6ea1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:40.742863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:40.743795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:40.744683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:40.744683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:40.744683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:40.744683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:40.744683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:40.744683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:40.744683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:40.744683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:40.744683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:29:40.744683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:40.751775 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 236/472] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:40.752992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea8ce3-d25a-4a2b-9ff5-edc1982c6ea1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:40.771747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:40.776597 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb430e-4019-488b-9fb7-8462a726ac68 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:40.825740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e43fa5-38e6-42df-a73a-a21a9438ab24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:40.826454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e43fa5-38e6-42df-a73a-a21a9438ab24 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:40.826762 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e43fa5-38e6-42df-a73a-a21a9438ab24 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:40.827471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e43fa5-38e6-42df-a73a-a21a9438ab24 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:40.828466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e43fa5-38e6-42df-a73a-a21a9438ab24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:40.828775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e43fa5-38e6-42df-a73a-a21a9438ab24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:40.830218 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e43fa5-38e6-42df-a73a-a21a9438ab24 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:40.850105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 237/473] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:40.850105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e43fa5-38e6-42df-a73a-a21a9438ab24 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:40.875638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea8ce3-d25a-4a2b-9ff5-edc1982c6ea1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:40.913606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3990b8-d983-4867-a2fb-e153fdd0f62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:40.914608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3990b8-d983-4867-a2fb-e153fdd0f62f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:40.915097 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3990b8-d983-4867-a2fb-e153fdd0f62f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:40.915557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3990b8-d983-4867-a2fb-e153fdd0f62f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:40.968189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e43fa5-38e6-42df-a73a-a21a9438ab24 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:40.993379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:41.004228 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47648fd1-c885-449c-8139-cdb9fccc4623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:41.004836 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47648fd1-c885-449c-8139-cdb9fccc4623 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:41.005648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47648fd1-c885-449c-8139-cdb9fccc4623 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:41.006012 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47648fd1-c885-449c-8139-cdb9fccc4623 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:41.007104 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47648fd1-c885-449c-8139-cdb9fccc4623 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:41.007408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47648fd1-c885-449c-8139-cdb9fccc4623 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:41.008817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47648fd1-c885-449c-8139-cdb9fccc4623 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:41.012537 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3990b8-d983-4867-a2fb-e153fdd0f62f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:41.014130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3990b8-d983-4867-a2fb-e153fdd0f62f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:41.029319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 238/474] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Oct 24 13:29:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:41.030020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47648fd1-c885-449c-8139-cdb9fccc4623 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:41.049308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:41.148242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47648fd1-c885-449c-8139-cdb9fccc4623 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:41.149946 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 237/475] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:29:41.151024 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3990b8-d983-4867-a2fb-e153fdd0f62f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:41.196014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b15c02-bd52-4aad-bb55-2921fdeed1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:41.197621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b15c02-bd52-4aad-bb55-2921fdeed1bd None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:41.198080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b15c02-bd52-4aad-bb55-2921fdeed1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:41.198539 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b15c02-bd52-4aad-bb55-2921fdeed1bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:41.199688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b15c02-bd52-4aad-bb55-2921fdeed1bd None admin] RBAC: Adding query filter params (role.id=037adbca67514eda861c68218a60c50a, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=d07f8443d5594f47bc81e9f9dc9ce43b) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:41.200702 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b15c02-bd52-4aad-bb55-2921fdeed1bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:41.202885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b15c02-bd52-4aad-bb55-2921fdeed1bd None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:41.211779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 239/476] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:29:41 2022] GET /identity/v3/role_assignments?user.id=d07f8443d5594f47bc81e9f9dc9ce43b&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=037adbca67514eda861c68218a60c50a => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:41.213314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b15c02-bd52-4aad-bb55-2921fdeed1bd None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:41.253153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:41.290270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3990b8-d983-4867-a2fb-e153fdd0f62f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:41.329202 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:41.356642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b15c02-bd52-4aad-bb55-2921fdeed1bd None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:41.371433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaecf5-dce3-47f8-a5d3-4a6559c958de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:41.372355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaecf5-dce3-47f8-a5d3-4a6559c958de None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:41.372652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaecf5-dce3-47f8-a5d3-4a6559c958de None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:41.373013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaecf5-dce3-47f8-a5d3-4a6559c958de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:41.408841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31438e3-c6f0-4ed8-b735-f36cc4e162ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:41.409726 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31438e3-c6f0-4ed8-b735-f36cc4e162ff None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:41.410150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31438e3-c6f0-4ed8-b735-f36cc4e162ff None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:41.410625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31438e3-c6f0-4ed8-b735-f36cc4e162ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:41.435086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31438e3-c6f0-4ed8-b735-f36cc4e162ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:41.438396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31438e3-c6f0-4ed8-b735-f36cc4e162ff None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-a31438e3-c6f0-4ed8-b735-f36cc4e162ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:41.475669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:41.476765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:41.477802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:41.477802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:41.477802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:41.484627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 240/477] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:41.490306 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31438e3-c6f0-4ed8-b735-f36cc4e162ff None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:41.517030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecaecf5-dce3-47f8-a5d3-4a6559c958de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:41.519364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecaecf5-dce3-47f8-a5d3-4a6559c958de None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:41.634123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:41.635586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31438e3-c6f0-4ed8-b735-f36cc4e162ff None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:41.653641 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 238/478] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:29:41.659023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaecf5-dce3-47f8-a5d3-4a6559c958de None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:41.683199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a1de0-f7b1-4926-8e11-4880b85a8ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:41.684353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a1de0-f7b1-4926-8e11-4880b85a8ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:41.684789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a1de0-f7b1-4926-8e11-4880b85a8ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:41.685379 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a1de0-f7b1-4926-8e11-4880b85a8ad4 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:41.686554 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a1de0-f7b1-4926-8e11-4880b85a8ad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:41.687260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a1de0-f7b1-4926-8e11-4880b85a8ad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:41.689287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:41.689978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a1de0-f7b1-4926-8e11-4880b85a8ad4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:41.709169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 241/479] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:41.714544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a1de0-f7b1-4926-8e11-4880b85a8ad4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:41.781595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaecf5-dce3-47f8-a5d3-4a6559c958de None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:41.831409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818901f-6aea-49e3-b82e-7e8f2360f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:41.832334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818901f-6aea-49e3-b82e-7e8f2360f6cd None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:41.832761 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818901f-6aea-49e3-b82e-7e8f2360f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:41.833289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818901f-6aea-49e3-b82e-7e8f2360f6cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:41.854754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818901f-6aea-49e3-b82e-7e8f2360f6cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:41.857840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818901f-6aea-49e3-b82e-7e8f2360f6cd None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:41.868998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a1de0-f7b1-4926-8e11-4880b85a8ad4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-d818901f-6aea-49e3-b82e-7e8f2360f6cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:41.895452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:41.896809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:41.897818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:41.897818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:41.897818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:41.897818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 239/480] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:41.902554 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818901f-6aea-49e3-b82e-7e8f2360f6cd None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:41.935970 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c9132-a294-4aa1-a3be-649be449758d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:41.936950 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c9132-a294-4aa1-a3be-649be449758d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:41.937434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c9132-a294-4aa1-a3be-649be449758d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:41.937926 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c9132-a294-4aa1-a3be-649be449758d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:41.952204 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c9132-a294-4aa1-a3be-649be449758d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:41.955900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c9132-a294-4aa1-a3be-649be449758d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-389c9132-a294-4aa1-a3be-649be449758d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:41.979249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:41.980175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:41.981008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:41.981008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:41.981008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:41.981008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:41.981008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:41.981008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:41.981008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:41.981008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:41.989707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 242/481] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:41.995053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c9132-a294-4aa1-a3be-649be449758d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:42.000069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:42.053813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:42.062746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818901f-6aea-49e3-b82e-7e8f2360f6cd None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:42.097450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467beeba-e444-4d56-b879-c64672d4c24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:42.098058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467beeba-e444-4d56-b879-c64672d4c24d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:42.098382 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467beeba-e444-4d56-b879-c64672d4c24d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:42.098651 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467beeba-e444-4d56-b879-c64672d4c24d None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:42.100386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467beeba-e444-4d56-b879-c64672d4c24d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:42.100691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467beeba-e444-4d56-b879-c64672d4c24d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:42.102051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467beeba-e444-4d56-b879-c64672d4c24d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:42.106956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c9132-a294-4aa1-a3be-649be449758d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:42.124670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 240/482] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:42.126006 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0277141-adbc-4d68-8160-ab2c7be4b8c2 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:42.126331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0277141-adbc-4d68-8160-ab2c7be4b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:42.126605 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0277141-adbc-4d68-8160-ab2c7be4b8c2 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:42.127842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 241/483] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:42.128525 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0277141-adbc-4d68-8160-ab2c7be4b8c2 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:42.144407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b77af5-38ca-4960-a83f-3d9b329be2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:42.145164 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b77af5-38ca-4960-a83f-3d9b329be2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:42.145544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b77af5-38ca-4960-a83f-3d9b329be2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:42.146544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b77af5-38ca-4960-a83f-3d9b329be2e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:42.147341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b77af5-38ca-4960-a83f-3d9b329be2e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:42.147746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b77af5-38ca-4960-a83f-3d9b329be2e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:42.149272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b77af5-38ca-4960-a83f-3d9b329be2e4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:42.163328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 243/484] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:42.164676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb0a27-25a4-4e2f-a448-5c84971ffd7f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:42.165062 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb0a27-25a4-4e2f-a448-5c84971ffd7f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:42.165435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb0a27-25a4-4e2f-a448-5c84971ffd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:42.184138 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-11cb0a27-25a4-4e2f-a448-5c84971ffd7f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:42.185526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-11cb0a27-25a4-4e2f-a448-5c84971ffd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:42.248334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0277141-adbc-4d68-8160-ab2c7be4b8c2 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:42.257909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 244/485] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:42.258939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb0a27-25a4-4e2f-a448-5c84971ffd7f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:42.259609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:42.304607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:42.322388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cfe40-3278-4a0b-84d6-515c7a0e24bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:42.323037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cfe40-3278-4a0b-84d6-515c7a0e24bb None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:42.323315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cfe40-3278-4a0b-84d6-515c7a0e24bb None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:42.323643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cfe40-3278-4a0b-84d6-515c7a0e24bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:42.335315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cfe40-3278-4a0b-84d6-515c7a0e24bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:42.337274 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cfe40-3278-4a0b-84d6-515c7a0e24bb None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-b83cfe40-3278-4a0b-84d6-515c7a0e24bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:42.350609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:42.351762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:42.352822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:42.352822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:42.352822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:42.352822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:42.352822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:42.352822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:42.352822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:42.352822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:42.352822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 242/486] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:42.359380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cfe40-3278-4a0b-84d6-515c7a0e24bb None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:42.416401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb0a27-25a4-4e2f-a448-5c84971ffd7f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:42.493279 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb3dc8-61b5-43f0-86d9-306267e555e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:42.493807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb3dc8-61b5-43f0-86d9-306267e555e3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:42.494084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb3dc8-61b5-43f0-86d9-306267e555e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:42.494376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb3dc8-61b5-43f0-86d9-306267e555e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:42.495101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb3dc8-61b5-43f0-86d9-306267e555e3 None admin] RBAC: Adding query filter params (role.id=037adbca67514eda861c68218a60c50a, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=5b8479c571ec449fbfc306d25fbf7fa2) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:42.495407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb3dc8-61b5-43f0-86d9-306267e555e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:42.496865 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb3dc8-61b5-43f0-86d9-306267e555e3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:42.514257 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 245/487] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:29:42 2022] GET /identity/v3/role_assignments?user.id=5b8479c571ec449fbfc306d25fbf7fa2&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=037adbca67514eda861c68218a60c50a => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:42.544394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cfe40-3278-4a0b-84d6-515c7a0e24bb None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:42.610969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6288bef-c95f-486c-bd3e-c57289f5efee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:42.611615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6288bef-c95f-486c-bd3e-c57289f5efee None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:42.611897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6288bef-c95f-486c-bd3e-c57289f5efee None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:42.612173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6288bef-c95f-486c-bd3e-c57289f5efee None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:42.612891 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6288bef-c95f-486c-bd3e-c57289f5efee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:42.613194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6288bef-c95f-486c-bd3e-c57289f5efee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:42.614834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6288bef-c95f-486c-bd3e-c57289f5efee None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:42.629528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb3dc8-61b5-43f0-86d9-306267e555e3 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:42.636586 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 243/488] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:42.648896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6288bef-c95f-486c-bd3e-c57289f5efee None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:42.698538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:42.763273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb3dc8-61b5-43f0-86d9-306267e555e3 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:42.770385 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:42.808247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae54585-0f0b-42ab-b6cf-082b26adba7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:42.808887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae54585-0f0b-42ab-b6cf-082b26adba7f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:42.809159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae54585-0f0b-42ab-b6cf-082b26adba7f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:42.809419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae54585-0f0b-42ab-b6cf-082b26adba7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:42.812658 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6288bef-c95f-486c-bd3e-c57289f5efee None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:42.823988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae54585-0f0b-42ab-b6cf-082b26adba7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-7ae54585-0f0b-42ab-b6cf-082b26adba7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:42.829603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:42.830541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:42.831338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 246/489] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:29:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:42.832182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d2c83-c789-4f39-a27b-5e1b2b798c36 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:42.832523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d2c83-c789-4f39-a27b-5e1b2b798c36 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:42.832789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d2c83-c789-4f39-a27b-5e1b2b798c36 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:42.838685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 247/490] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:42.840017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d2c83-c789-4f39-a27b-5e1b2b798c36 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:42.864340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b872fb-a006-44c7-88f0-53673e4b32c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:42.865529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b872fb-a006-44c7-88f0-53673e4b32c3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:42.865812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b872fb-a006-44c7-88f0-53673e4b32c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:42.866081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b872fb-a006-44c7-88f0-53673e4b32c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:42.867246 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b872fb-a006-44c7-88f0-53673e4b32c3 None admin] RBAC: Adding query filter params (role.id=037adbca67514eda861c68218a60c50a, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=3c8c594699c045a09c835910d894896b) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:42.867553 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b872fb-a006-44c7-88f0-53673e4b32c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:42.869089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b872fb-a006-44c7-88f0-53673e4b32c3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:42.876525 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 244/491] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:29:42 2022] GET /identity/v3/role_assignments?user.id=3c8c594699c045a09c835910d894896b&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=037adbca67514eda861c68218a60c50a => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:42.878453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65bed5-7d29-453a-98b5-a9410d03bfbd None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:42.878736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65bed5-7d29-453a-98b5-a9410d03bfbd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:42.879006 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65bed5-7d29-453a-98b5-a9410d03bfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:42.903124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ee65bed5-7d29-453a-98b5-a9410d03bfbd None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:42.904376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ee65bed5-7d29-453a-98b5-a9410d03bfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:43.021876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d2c83-c789-4f39-a27b-5e1b2b798c36 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:43.028812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 245/492] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:43.076145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc8494-e31b-43da-baff-c9a24bfecf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:43.076754 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc8494-e31b-43da-baff-c9a24bfecf50 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:43.077029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc8494-e31b-43da-baff-c9a24bfecf50 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:43.077296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc8494-e31b-43da-baff-c9a24bfecf50 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:43.077950 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc8494-e31b-43da-baff-c9a24bfecf50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:43.078272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc8494-e31b-43da-baff-c9a24bfecf50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:43.079760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc8494-e31b-43da-baff-c9a24bfecf50 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:43.089267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 248/493] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:43.094782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65bed5-7d29-453a-98b5-a9410d03bfbd None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:43.111809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:43.207757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65bed5-7d29-453a-98b5-a9410d03bfbd None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:43.247149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03fc00-17cf-4b9d-b3cf-b93f92fdbe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:43.247780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03fc00-17cf-4b9d-b3cf-b93f92fdbe3c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:43.248057 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03fc00-17cf-4b9d-b3cf-b93f92fdbe3c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:43.248325 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03fc00-17cf-4b9d-b3cf-b93f92fdbe3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:43.259141 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03fc00-17cf-4b9d-b3cf-b93f92fdbe3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:43.260572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03fc00-17cf-4b9d-b3cf-b93f92fdbe3c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-cc03fc00-17cf-4b9d-b3cf-b93f92fdbe3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:43.277729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:43.279086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:43.280175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:43.280175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:43.280175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:43.280175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 246/494] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:43.284879 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc8494-e31b-43da-baff-c9a24bfecf50 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:43.320450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:43.377299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03fc00-17cf-4b9d-b3cf-b93f92fdbe3c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:43.420926 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc8494-e31b-43da-baff-c9a24bfecf50 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:43.488785 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:43.493365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845c6be-6516-47a4-95e1-11e2c9346740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:43.493984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845c6be-6516-47a4-95e1-11e2c9346740 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:43.494295 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845c6be-6516-47a4-95e1-11e2c9346740 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:43.494613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845c6be-6516-47a4-95e1-11e2c9346740 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:43.495267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845c6be-6516-47a4-95e1-11e2c9346740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:43.495693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845c6be-6516-47a4-95e1-11e2c9346740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:43.497299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845c6be-6516-47a4-95e1-11e2c9346740 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:43.522445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 249/495] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:43.529956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845c6be-6516-47a4-95e1-11e2c9346740 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:43.558293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03fc00-17cf-4b9d-b3cf-b93f92fdbe3c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:43.626468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbf272-f789-4657-b0dc-64cfa6136cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:43.627136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbf272-f789-4657-b0dc-64cfa6136cdf None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:43.627422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbf272-f789-4657-b0dc-64cfa6136cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:43.627697 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbf272-f789-4657-b0dc-64cfa6136cdf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:43.680397 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbf272-f789-4657-b0dc-64cfa6136cdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-48cbf272-f789-4657-b0dc-64cfa6136cdf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:43.690231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:43.691203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:43.692117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 247/496] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:43.701073 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbf272-f789-4657-b0dc-64cfa6136cdf None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:43.787390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:43.824263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845c6be-6516-47a4-95e1-11e2c9346740 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:43.946461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c900-79ad-4baf-8168-4b6d3d9d49f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:43.947116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c900-79ad-4baf-8168-4b6d3d9d49f8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:43.947393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c900-79ad-4baf-8168-4b6d3d9d49f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:43.947660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c900-79ad-4baf-8168-4b6d3d9d49f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:43.950316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbf272-f789-4657-b0dc-64cfa6136cdf None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:43.964317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555c900-79ad-4baf-8168-4b6d3d9d49f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:43.965751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555c900-79ad-4baf-8168-4b6d3d9d49f8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-6555c900-79ad-4baf-8168-4b6d3d9d49f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:44.000981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:44.001991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:44.002968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:44.002968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:44.002968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:44.002968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:44.002968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:44.002968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:44.002968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:44.002968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:44.002968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:29:44.002968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:44.002968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 250/497] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Oct 24 13:29:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 473 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:44.014134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c900-79ad-4baf-8168-4b6d3d9d49f8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:44.030552 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:44.051155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80849590-408f-484f-be83-86a2b6cb4abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:44.051786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80849590-408f-484f-be83-86a2b6cb4abe None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:44.052677 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80849590-408f-484f-be83-86a2b6cb4abe None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:44.053009 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80849590-408f-484f-be83-86a2b6cb4abe None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:44.053683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80849590-408f-484f-be83-86a2b6cb4abe None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:44.053983 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80849590-408f-484f-be83-86a2b6cb4abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:44.055383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80849590-408f-484f-be83-86a2b6cb4abe None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:44.079713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 248/498] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:44.080977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e7e5e-c696-412d-9505-49ec797155ad None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:44.086199 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e7e5e-c696-412d-9505-49ec797155ad None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:44.086199 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e7e5e-c696-412d-9505-49ec797155ad None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:44.087474 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 249/499] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:44.088769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e7e5e-c696-412d-9505-49ec797155ad None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:44.226898 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e7e5e-c696-412d-9505-49ec797155ad None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:44.233625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c900-79ad-4baf-8168-4b6d3d9d49f8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:44.264141 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263416f-6dd1-471f-a51f-dc13b794522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:44.264924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263416f-6dd1-471f-a51f-dc13b794522d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:44.265329 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263416f-6dd1-471f-a51f-dc13b794522d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:44.265699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263416f-6dd1-471f-a51f-dc13b794522d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:44.278853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263416f-6dd1-471f-a51f-dc13b794522d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:44.280837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263416f-6dd1-471f-a51f-dc13b794522d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:44.288342 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429822dd-b590-4d9a-aa04-facf5679aa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:44.290010 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429822dd-b590-4d9a-aa04-facf5679aa6f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:44.290895 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429822dd-b590-4d9a-aa04-facf5679aa6f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:44.291833 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429822dd-b590-4d9a-aa04-facf5679aa6f None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:44.293023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429822dd-b590-4d9a-aa04-facf5679aa6f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:44.293527 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429822dd-b590-4d9a-aa04-facf5679aa6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:44.295186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429822dd-b590-4d9a-aa04-facf5679aa6f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-9263416f-6dd1-471f-a51f-dc13b794522d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:44.297638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:44.298577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:44.300317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:44.300317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:44.300317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:44.300317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 250/500] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:44.300881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aff65-7028-4dbf-9e69-3cd9f8474103 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:44.302007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aff65-7028-4dbf-9e69-3cd9f8474103 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:44.302007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aff65-7028-4dbf-9e69-3cd9f8474103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:44.320681 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 251/501] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Oct 24 13:29:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:44.321790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429822dd-b590-4d9a-aa04-facf5679aa6f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:44.326241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ce2aff65-7028-4dbf-9e69-3cd9f8474103 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:44.327194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ce2aff65-7028-4dbf-9e69-3cd9f8474103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:44.332148 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:44.365472 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:44.410529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 251/502] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:44.412512 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2aff65-7028-4dbf-9e69-3cd9f8474103 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:44.452705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429822dd-b590-4d9a-aa04-facf5679aa6f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:44.495571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da563910-15da-4271-838d-63b616355e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:44.496506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da563910-15da-4271-838d-63b616355e46 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:44.496965 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da563910-15da-4271-838d-63b616355e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:44.497415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da563910-15da-4271-838d-63b616355e46 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:44.498812 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da563910-15da-4271-838d-63b616355e46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:44.499294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da563910-15da-4271-838d-63b616355e46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:44.500818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da563910-15da-4271-838d-63b616355e46 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:44.516155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 252/503] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:44.521228 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da563910-15da-4271-838d-63b616355e46 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:44.633753 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:44.642613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2aff65-7028-4dbf-9e69-3cd9f8474103 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:44.674479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da563910-15da-4271-838d-63b616355e46 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:44.727923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574222ef-e255-457f-838b-aff85e3912f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:44.729637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574222ef-e255-457f-838b-aff85e3912f8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:44.731009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574222ef-e255-457f-838b-aff85e3912f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:44.731982 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574222ef-e255-457f-838b-aff85e3912f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:44.732761 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a38c1e-f4a2-446f-bbc7-67c354b7bb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:44.734322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a38c1e-f4a2-446f-bbc7-67c354b7bb84 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:44.736078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a38c1e-f4a2-446f-bbc7-67c354b7bb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:44.736372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a38c1e-f4a2-446f-bbc7-67c354b7bb84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:44.744995 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574222ef-e255-457f-838b-aff85e3912f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:44.746465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574222ef-e255-457f-838b-aff85e3912f8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:44.751748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a38c1e-f4a2-446f-bbc7-67c354b7bb84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:44.753392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a38c1e-f4a2-446f-bbc7-67c354b7bb84 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-574222ef-e255-457f-838b-aff85e3912f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:44.760166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:44.761107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:44.762022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:44.762022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:44.762022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:44.762022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:44.762022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:44.762022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:44.762022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:44.762022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:44.762022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:29:44.762022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:44.762022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 253/504] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:29:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:44.762624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574222ef-e255-457f-838b-aff85e3912f8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-b5a38c1e-f4a2-446f-bbc7-67c354b7bb84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:44.772745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:44.773713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:44.774737 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:44.774737 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:44.774737 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:44.776582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 252/505] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:44.777649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a38c1e-f4a2-446f-bbc7-67c354b7bb84 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:44.806226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:44.889685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a38c1e-f4a2-446f-bbc7-67c354b7bb84 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:44.896753 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574222ef-e255-457f-838b-aff85e3912f8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:44.930930 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:44.932478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec97cb-c3ea-4340-b231-660d9e428160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:44.933117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec97cb-c3ea-4340-b231-660d9e428160 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:44.933387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec97cb-c3ea-4340-b231-660d9e428160 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:44.933658 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec97cb-c3ea-4340-b231-660d9e428160 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:44.935004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec97cb-c3ea-4340-b231-660d9e428160 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:44.935317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec97cb-c3ea-4340-b231-660d9e428160 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:44.937243 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec97cb-c3ea-4340-b231-660d9e428160 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:44.945727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2df47c-81c2-4710-bf4e-68573ea19c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:44.946808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2df47c-81c2-4710-bf4e-68573ea19c63 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:44.947286 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2df47c-81c2-4710-bf4e-68573ea19c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:44.947751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2df47c-81c2-4710-bf4e-68573ea19c63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:44.960725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2df47c-81c2-4710-bf4e-68573ea19c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:44.962013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 253/506] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Oct 24 13:29:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:44.962963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec97cb-c3ea-4340-b231-660d9e428160 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:44.963469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2df47c-81c2-4710-bf4e-68573ea19c63 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-6d2df47c-81c2-4710-bf4e-68573ea19c63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:44.979318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:44.980275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:44.981128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:44.981128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:44.981128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:44.987982 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 254/507] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:44.989158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2df47c-81c2-4710-bf4e-68573ea19c63 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:45.067577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:45.089886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec97cb-c3ea-4340-b231-660d9e428160 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:45.092716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2df47c-81c2-4710-bf4e-68573ea19c63 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:45.126927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66975af7-785f-4612-b4a5-d6e7c29cb71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:45.127278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3056a92-6fda-4122-a972-a664960fdf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:45.127892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056a92-6fda-4122-a972-a664960fdf26 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.128174 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056a92-6fda-4122-a972-a664960fdf26 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.128448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056a92-6fda-4122-a972-a664960fdf26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.128615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66975af7-785f-4612-b4a5-d6e7c29cb71a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.129074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66975af7-785f-4612-b4a5-d6e7c29cb71a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.129416 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66975af7-785f-4612-b4a5-d6e7c29cb71a None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.130690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66975af7-785f-4612-b4a5-d6e7c29cb71a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:45.130995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66975af7-785f-4612-b4a5-d6e7c29cb71a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:45.132362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66975af7-785f-4612-b4a5-d6e7c29cb71a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:45.143959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 254/508] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:45.147064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66975af7-785f-4612-b4a5-d6e7c29cb71a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:45.181282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:45.209114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3056a92-6fda-4122-a972-a664960fdf26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:45.211629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3056a92-6fda-4122-a972-a664960fdf26 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:45.261352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66975af7-785f-4612-b4a5-d6e7c29cb71a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:45.297141 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98442de-130a-495d-9ab5-b98ff3b88ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:45.297750 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98442de-130a-495d-9ab5-b98ff3b88ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.298023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98442de-130a-495d-9ab5-b98ff3b88ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.298321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98442de-130a-495d-9ab5-b98ff3b88ce6 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.298954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98442de-130a-495d-9ab5-b98ff3b88ce6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:45.299248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98442de-130a-495d-9ab5-b98ff3b88ce6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:45.300597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98442de-130a-495d-9ab5-b98ff3b88ce6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:45.310919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:45.312684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 255/509] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:29:45.314029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a567bcb-9238-4122-a779-2ba2b40d07d7 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.314344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a567bcb-9238-4122-a779-2ba2b40d07d7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.314616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a567bcb-9238-4122-a779-2ba2b40d07d7 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.315987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 256/510] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:45.316592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a567bcb-9238-4122-a779-2ba2b40d07d7 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:45.324862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 255/511] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:45.324862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98442de-130a-495d-9ab5-b98ff3b88ce6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:45.408348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:45.425159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98442de-130a-495d-9ab5-b98ff3b88ce6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:45.433947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a567bcb-9238-4122-a779-2ba2b40d07d7 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:45.464930 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec3ba8-a65a-4dd9-9473-bf91ff827bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:45.466076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3ba8-a65a-4dd9-9473-bf91ff827bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.466546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3ba8-a65a-4dd9-9473-bf91ff827bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.466959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3ba8-a65a-4dd9-9473-bf91ff827bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.471104 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15939703-7ae0-4141-9506-bf583fa13f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:45.471764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939703-7ae0-4141-9506-bf583fa13f3b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.472043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939703-7ae0-4141-9506-bf583fa13f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.472314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939703-7ae0-4141-9506-bf583fa13f3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.479707 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec3ba8-a65a-4dd9-9473-bf91ff827bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:45.481505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec3ba8-a65a-4dd9-9473-bf91ff827bd8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:45.481963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15939703-7ae0-4141-9506-bf583fa13f3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:45.483416 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15939703-7ae0-4141-9506-bf583fa13f3b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-15939703-7ae0-4141-9506-bf583fa13f3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:45.496487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:45.497456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:45.498352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:45.498352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:45.498352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:45.498352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:45.498352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:45.498352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:45.498352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:45.498352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:45.498352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 257/512] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:45.499197 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b1beb-1cc0-4706-9f20-0016a4e874ba None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.499197 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b1beb-1cc0-4706-9f20-0016a4e874ba None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.499429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b1beb-1cc0-4706-9f20-0016a4e874ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-4bec3ba8-a65a-4dd9-9473-bf91ff827bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:45.502836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:45.503771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:45.504620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:45.504620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:45.504620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:45.505165 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 256/513] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:45.505855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec3ba8-a65a-4dd9-9473-bf91ff827bd8 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:45.519028 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-eb0b1beb-1cc0-4706-9f20-0016a4e874ba None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:45.520214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-eb0b1beb-1cc0-4706-9f20-0016a4e874ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:45.568261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:45.593466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 258/514] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:45.594401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b1beb-1cc0-4706-9f20-0016a4e874ba None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:45.619264 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec3ba8-a65a-4dd9-9473-bf91ff827bd8 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:45.653769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a888f-53f6-4b4b-b8aa-81671e6ce027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:45.654391 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a888f-53f6-4b4b-b8aa-81671e6ce027 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.654676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a888f-53f6-4b4b-b8aa-81671e6ce027 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.654947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a888f-53f6-4b4b-b8aa-81671e6ce027 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.655584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a888f-53f6-4b4b-b8aa-81671e6ce027 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:45.657013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a888f-53f6-4b4b-b8aa-81671e6ce027 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:45.665392 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:45.686191 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 257/515] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:29:45 2022] POST /identity/v3/users => generated 254 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:45.686191 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a888f-53f6-4b4b-b8aa-81671e6ce027 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:45.715001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b1beb-1cc0-4706-9f20-0016a4e874ba None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:45.754729 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8b96d-12a8-47a5-81a6-df2aec09aaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:45.756267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8b96d-12a8-47a5-81a6-df2aec09aaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.756612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8b96d-12a8-47a5-81a6-df2aec09aaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.756955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8b96d-12a8-47a5-81a6-df2aec09aaa1 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.757741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8b96d-12a8-47a5-81a6-df2aec09aaa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:45.758140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8b96d-12a8-47a5-81a6-df2aec09aaa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:45.759786 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8b96d-12a8-47a5-81a6-df2aec09aaa1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:45.778345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 259/516] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:45.780372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14a371-d04b-457b-af99-fd531e6c764c None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.780850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14a371-d04b-457b-af99-fd531e6c764c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.781299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14a371-d04b-457b-af99-fd531e6c764c None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.783241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 260/517] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:45.784944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14a371-d04b-457b-af99-fd531e6c764c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:45.828617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a888f-53f6-4b4b-b8aa-81671e6ce027 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:45.843316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:45.882892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5b9bc-19ae-47a6-9f85-5b8c5cd17c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:45.883523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5b9bc-19ae-47a6-9f85-5b8c5cd17c72 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.883804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5b9bc-19ae-47a6-9f85-5b8c5cd17c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.884072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5b9bc-19ae-47a6-9f85-5b8c5cd17c72 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.885099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5b9bc-19ae-47a6-9f85-5b8c5cd17c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:45.885395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5b9bc-19ae-47a6-9f85-5b8c5cd17c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:45.887130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5b9bc-19ae-47a6-9f85-5b8c5cd17c72 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:45.900981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 258/518] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:45.906242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1888f16-0269-4354-aba7-5bae9e677b8f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.906665 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1888f16-0269-4354-aba7-5bae9e677b8f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.906946 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1888f16-0269-4354-aba7-5bae9e677b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.913552 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14a371-d04b-457b-af99-fd531e6c764c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:45.927252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c1888f16-0269-4354-aba7-5bae9e677b8f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:45.928385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c1888f16-0269-4354-aba7-5bae9e677b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:45.943582 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:45.954597 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0300f-86c6-4878-8d9d-fb695dcc02a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:45.955262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0300f-86c6-4878-8d9d-fb695dcc02a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:45.955543 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0300f-86c6-4878-8d9d-fb695dcc02a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:45.955816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0300f-86c6-4878-8d9d-fb695dcc02a4 None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/18bb93669846435a85eb34771cfb09fe/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:45.958645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0300f-86c6-4878-8d9d-fb695dcc02a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=18bb93669846435a85eb34771cfb09fe, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:45.960353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0300f-86c6-4878-8d9d-fb695dcc02a4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:45.986194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 261/519] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:29:45 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/18bb93669846435a85eb34771cfb09fe/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:45.986194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0300f-86c6-4878-8d9d-fb695dcc02a4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:46.002899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 259/520] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:46.003844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1888f16-0269-4354-aba7-5bae9e677b8f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:46.195009 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:46.202888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1888f16-0269-4354-aba7-5bae9e677b8f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:46.205267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0300f-86c6-4878-8d9d-fb695dcc02a4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:46.254759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e78a2e-5d61-4bf6-8755-ea32221de36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:46.256128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e986f-7c76-4dc2-8411-53ea08ff76e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:46.256276 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78a2e-5d61-4bf6-8755-ea32221de36a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:46.256443 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78a2e-5d61-4bf6-8755-ea32221de36a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:46.256694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78a2e-5d61-4bf6-8755-ea32221de36a None admin] PATH_INFO: `/v3/services/network` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:46.257277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e78a2e-5d61-4bf6-8755-ea32221de36a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:46.257601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e986f-7c76-4dc2-8411-53ea08ff76e9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:46.258041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e986f-7c76-4dc2-8411-53ea08ff76e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:46.258507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e986f-7c76-4dc2-8411-53ea08ff76e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:46.258677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e78a2e-5d61-4bf6-8755-ea32221de36a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:46.266965 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e986f-7c76-4dc2-8411-53ea08ff76e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:46.268595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e986f-7c76-4dc2-8411-53ea08ff76e9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-97e78a2e-5d61-4bf6-8755-ea32221de36a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:46.278978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:46.280027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:46.280875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 262/521] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:45 2022] GET /identity/v3/services/network => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-a88e986f-7c76-4dc2-8411-53ea08ff76e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:46.284127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:46.284988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:46.285848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:46.285848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:46.285848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:46.285848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:46.285848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:46.285848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:46.285848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:46.285848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:46.286829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e78a2e-5d61-4bf6-8755-ea32221de36a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:46.288388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 260/522] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:46.295471 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e986f-7c76-4dc2-8411-53ea08ff76e9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:46.372807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:46.428433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e986f-7c76-4dc2-8411-53ea08ff76e9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:46.434867 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e78a2e-5d61-4bf6-8755-ea32221de36a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:46.469184 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c7fe4-31cb-4c78-8832-ff8ace23bd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:46.469755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c7fe4-31cb-4c78-8832-ff8ace23bd95 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:46.469990 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c7fe4-31cb-4c78-8832-ff8ace23bd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:46.470289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c7fe4-31cb-4c78-8832-ff8ace23bd95 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:46.471205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c7fe4-31cb-4c78-8832-ff8ace23bd95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:46.471560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c7fe4-31cb-4c78-8832-ff8ace23bd95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:46.472450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff9570-7a56-4c49-8b31-8af4c2ee30c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:46.472987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c7fe4-31cb-4c78-8832-ff8ace23bd95 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:46.473383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff9570-7a56-4c49-8b31-8af4c2ee30c0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:46.473851 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff9570-7a56-4c49-8b31-8af4c2ee30c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:46.474424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff9570-7a56-4c49-8b31-8af4c2ee30c0 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:46.475191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff9570-7a56-4c49-8b31-8af4c2ee30c0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:46.475630 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff9570-7a56-4c49-8b31-8af4c2ee30c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:46.477135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff9570-7a56-4c49-8b31-8af4c2ee30c0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:46.482398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 263/523] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:29:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:46.483475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff9570-7a56-4c49-8b31-8af4c2ee30c0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:46.497910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 261/524] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:46.497910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c7fe4-31cb-4c78-8832-ff8ace23bd95 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:46.522636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:46.595991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff9570-7a56-4c49-8b31-8af4c2ee30c0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:46.613524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:46.616766 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c7fe4-31cb-4c78-8832-ff8ace23bd95 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:46.633522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d965f4-68c8-4570-95cb-eef25fdd60fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:46.634472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d965f4-68c8-4570-95cb-eef25fdd60fd None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:46.634612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d965f4-68c8-4570-95cb-eef25fdd60fd None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:46.634801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d965f4-68c8-4570-95cb-eef25fdd60fd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:46.635380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d965f4-68c8-4570-95cb-eef25fdd60fd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:46.636736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d965f4-68c8-4570-95cb-eef25fdd60fd None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-e3d965f4-68c8-4570-95cb-eef25fdd60fd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:46.646205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:46.647486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:46.648339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 264/525] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Oct 24 13:29:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:46.649149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d965f4-68c8-4570-95cb-eef25fdd60fd None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:46.666773 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82054e3-1cd0-4355-8a73-231f7104de22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:46.667712 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82054e3-1cd0-4355-8a73-231f7104de22 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:46.668170 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82054e3-1cd0-4355-8a73-231f7104de22 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:46.668591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82054e3-1cd0-4355-8a73-231f7104de22 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:46.669388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82054e3-1cd0-4355-8a73-231f7104de22 None admin] RBAC: Adding query filter params (type=network) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:46.670514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82054e3-1cd0-4355-8a73-231f7104de22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:46.672161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82054e3-1cd0-4355-8a73-231f7104de22 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:46.686738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 262/526] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:29:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:46.687939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82054e3-1cd0-4355-8a73-231f7104de22 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:46.806569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:46.840353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d965f4-68c8-4570-95cb-eef25fdd60fd None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:46.841867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82054e3-1cd0-4355-8a73-231f7104de22 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:46.879619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ca211-a89f-43fe-9e46-0d48804fdd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:46.880780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca211-a89f-43fe-9e46-0d48804fdd6a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:46.881060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca211-a89f-43fe-9e46-0d48804fdd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:46.881333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca211-a89f-43fe-9e46-0d48804fdd6a None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:46.881987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ca211-a89f-43fe-9e46-0d48804fdd6a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:46.882346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ca211-a89f-43fe-9e46-0d48804fdd6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:46.883749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ca211-a89f-43fe-9e46-0d48804fdd6a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:46.889424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 263/527] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Oct 24 13:29:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:46.895195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ca211-a89f-43fe-9e46-0d48804fdd6a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:46.917845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed083e01-4d6e-4bbc-9027-9c2c0e29d994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:46.918600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed083e01-4d6e-4bbc-9027-9c2c0e29d994 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:46.918874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed083e01-4d6e-4bbc-9027-9c2c0e29d994 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:46.919140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed083e01-4d6e-4bbc-9027-9c2c0e29d994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:46.919859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed083e01-4d6e-4bbc-9027-9c2c0e29d994 None admin] RBAC: Adding query filter params (role.id=037adbca67514eda861c68218a60c50a, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=0ea0d300a51c41fba4d4193dd2251a50) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:46.920164 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed083e01-4d6e-4bbc-9027-9c2c0e29d994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:46.921697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed083e01-4d6e-4bbc-9027-9c2c0e29d994 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:46.929451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 265/528] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:29:46 2022] GET /identity/v3/role_assignments?user.id=0ea0d300a51c41fba4d4193dd2251a50&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=037adbca67514eda861c68218a60c50a => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:47.011074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ca211-a89f-43fe-9e46-0d48804fdd6a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:47.051568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffe377-486f-4ab4-a24a-981ba7b2ee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:47.052771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffe377-486f-4ab4-a24a-981ba7b2ee33 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:47.053048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffe377-486f-4ab4-a24a-981ba7b2ee33 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:47.053319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffe377-486f-4ab4-a24a-981ba7b2ee33 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:47.053960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffe377-486f-4ab4-a24a-981ba7b2ee33 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:47.054288 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffe377-486f-4ab4-a24a-981ba7b2ee33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:47.055893 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffe377-486f-4ab4-a24a-981ba7b2ee33 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:47.062079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 264/529] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Oct 24 13:29:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:48.109936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a41aa-f5e8-4783-8c65-484820e8cca0 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:48.110490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a41aa-f5e8-4783-8c65-484820e8cca0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:48.110490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a41aa-f5e8-4783-8c65-484820e8cca0 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:48.112130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 266/530] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:48.118224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f731dc-b165-46eb-a7e8-0763ce4b0a7c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:48.119103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f731dc-b165-46eb-a7e8-0763ce4b0a7c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:48.119575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f731dc-b165-46eb-a7e8-0763ce4b0a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:48.138442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-89f731dc-b165-46eb-a7e8-0763ce4b0a7c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:48.139945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-89f731dc-b165-46eb-a7e8-0763ce4b0a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:48.143065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:48.226266 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 265/531] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:48.508891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a41aa-f5e8-4783-8c65-484820e8cca0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:48.545372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9b877-ad8a-4da0-a295-a46c3f4b380f None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:48.546031 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9b877-ad8a-4da0-a295-a46c3f4b380f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:48.546542 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9b877-ad8a-4da0-a295-a46c3f4b380f None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:48.548307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 266/532] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:48.554357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a401a3-f6ca-4855-9f68-6131f82e7aa6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:48.554917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a401a3-f6ca-4855-9f68-6131f82e7aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:48.555400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a401a3-f6ca-4855-9f68-6131f82e7aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:48.559817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:48.575054 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b2a401a3-f6ca-4855-9f68-6131f82e7aa6 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:48.576326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b2a401a3-f6ca-4855-9f68-6131f82e7aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:48.644233 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a41aa-f5e8-4783-8c65-484820e8cca0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:48.695228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 267/533] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:29:48.712201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01ccf3-8866-465c-970d-0148c8e7c8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:48.712838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01ccf3-8866-465c-970d-0148c8e7c8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:48.713111 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01ccf3-8866-465c-970d-0148c8e7c8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:48.713374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01ccf3-8866-465c-970d-0148c8e7c8b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:48.729375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01ccf3-8866-465c-970d-0148c8e7c8b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-2c01ccf3-8866-465c-970d-0148c8e7c8b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:48.734939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:48.735840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:48.736665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:48.742267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 267/534] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:48.746023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a401a3-f6ca-4855-9f68-6131f82e7aa6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:48.836874 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:48.873236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a401a3-f6ca-4855-9f68-6131f82e7aa6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:48.907561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b8340-a0f5-4705-8c35-345743f82bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:48.908538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b8340-a0f5-4705-8c35-345743f82bca None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:48.909030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b8340-a0f5-4705-8c35-345743f82bca None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:48.909492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b8340-a0f5-4705-8c35-345743f82bca None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:48.910496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b8340-a0f5-4705-8c35-345743f82bca None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:48.911002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b8340-a0f5-4705-8c35-345743f82bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:48.913192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b8340-a0f5-4705-8c35-345743f82bca None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:48.926262 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 268/535] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:48.928246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01ccf3-8866-465c-970d-0148c8e7c8b8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:49.069117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01ccf3-8866-465c-970d-0148c8e7c8b8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:49.071504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:49.072753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b8340-a0f5-4705-8c35-345743f82bca None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:49.124067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94d9f9-ad26-40a9-bcff-7edfaec47f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:49.125405 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d9f9-ad26-40a9-bcff-7edfaec47f4a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:49.125887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d9f9-ad26-40a9-bcff-7edfaec47f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:49.126462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94d9f9-ad26-40a9-bcff-7edfaec47f4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:49.149440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94d9f9-ad26-40a9-bcff-7edfaec47f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:49.151327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94d9f9-ad26-40a9-bcff-7edfaec47f4a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-9f94d9f9-ad26-40a9-bcff-7edfaec47f4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:49.174622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:49.175619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:49.176498 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:49.176498 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:49.176498 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:49.183572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 268/536] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:49.185600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94d9f9-ad26-40a9-bcff-7edfaec47f4a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:49.288931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b8340-a0f5-4705-8c35-345743f82bca None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:49.306251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94d9f9-ad26-40a9-bcff-7edfaec47f4a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:49.343182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357db20d-03b0-41ed-a38a-661594ec6d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:49.344173 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357db20d-03b0-41ed-a38a-661594ec6d38 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:49.344675 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357db20d-03b0-41ed-a38a-661594ec6d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:49.345161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357db20d-03b0-41ed-a38a-661594ec6d38 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:49.346219 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357db20d-03b0-41ed-a38a-661594ec6d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:49.346754 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357db20d-03b0-41ed-a38a-661594ec6d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:49.348406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357db20d-03b0-41ed-a38a-661594ec6d38 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:49.366345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 269/537] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:49.369532 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357db20d-03b0-41ed-a38a-661594ec6d38 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:49.378592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:49.389439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:49.403861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99fb1f-beb9-45e6-8672-aa40a6cfd9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:49.404460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99fb1f-beb9-45e6-8672-aa40a6cfd9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:49.404806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99fb1f-beb9-45e6-8672-aa40a6cfd9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:49.404995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99fb1f-beb9-45e6-8672-aa40a6cfd9a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:49.443084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99fb1f-beb9-45e6-8672-aa40a6cfd9a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-cf99fb1f-beb9-45e6-8672-aa40a6cfd9a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:49.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:49.449256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:49.455002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 269/538] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:29:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:49.471435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99fb1f-beb9-45e6-8672-aa40a6cfd9a9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:49.491358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357db20d-03b0-41ed-a38a-661594ec6d38 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:49.538679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a675b8-e5a9-40dc-b343-080285dad759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:49.540838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a675b8-e5a9-40dc-b343-080285dad759 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:49.542411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a675b8-e5a9-40dc-b343-080285dad759 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:49.543546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a675b8-e5a9-40dc-b343-080285dad759 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:49.561658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a675b8-e5a9-40dc-b343-080285dad759 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:49.566320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a675b8-e5a9-40dc-b343-080285dad759 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b2a675b8-e5a9-40dc-b343-080285dad759 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:49.596720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:49.597718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:49.599282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:49.599282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:49.599282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:49.599282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:49.599282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:49.599282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:49.599282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:49.599282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:49.599282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:29:49.599282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:49.606839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 270/539] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Oct 24 13:29:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:49.610800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a675b8-e5a9-40dc-b343-080285dad759 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:49.695931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99fb1f-beb9-45e6-8672-aa40a6cfd9a9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:49.731954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4137d8-d02a-4e24-a38a-929d21c2fd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:49.732648 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4137d8-d02a-4e24-a38a-929d21c2fd78 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:49.732902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4137d8-d02a-4e24-a38a-929d21c2fd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:49.733171 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4137d8-d02a-4e24-a38a-929d21c2fd78 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:49.733830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4137d8-d02a-4e24-a38a-929d21c2fd78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:49.734141 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4137d8-d02a-4e24-a38a-929d21c2fd78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:49.735512 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4137d8-d02a-4e24-a38a-929d21c2fd78 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:49.743239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:49.744723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 270/540] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:49.745815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9398b-aa5f-47e1-8556-3968b7c49732 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:49.746103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9398b-aa5f-47e1-8556-3968b7c49732 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:49.746405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9398b-aa5f-47e1-8556-3968b7c49732 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:49.750294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 271/541] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:49.752047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c478b-4989-4346-862b-d3b036d6318f None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:49.752332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c478b-4989-4346-862b-d3b036d6318f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:49.752600 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c478b-4989-4346-862b-d3b036d6318f None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:49.754545 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 272/542] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:49.759774 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c478b-4989-4346-862b-d3b036d6318f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:49.775172 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a675b8-e5a9-40dc-b343-080285dad759 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:49.810009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7631c87-c0bc-4d46-8a9f-88e72e9be98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:49.810654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7631c87-c0bc-4d46-8a9f-88e72e9be98e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:49.810928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7631c87-c0bc-4d46-8a9f-88e72e9be98e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:49.811192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7631c87-c0bc-4d46-8a9f-88e72e9be98e None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:49.811911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7631c87-c0bc-4d46-8a9f-88e72e9be98e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:49.812208 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7631c87-c0bc-4d46-8a9f-88e72e9be98e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:49.813622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7631c87-c0bc-4d46-8a9f-88e72e9be98e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:49.826546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:49.832429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 271/543] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Oct 24 13:29:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:49.838950 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716d785-db78-4bd3-a7d0-6a884f752e6d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:49.839257 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716d785-db78-4bd3-a7d0-6a884f752e6d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:49.839507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716d785-db78-4bd3-a7d0-6a884f752e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:49.859702 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5716d785-db78-4bd3-a7d0-6a884f752e6d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:49.860922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5716d785-db78-4bd3-a7d0-6a884f752e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:49.886150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c478b-4989-4346-862b-d3b036d6318f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:49.925139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1ab0d-db9a-417b-8303-f371f3523079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:49.925779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1ab0d-db9a-417b-8303-f371f3523079 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:49.926058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1ab0d-db9a-417b-8303-f371f3523079 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:49.926357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1ab0d-db9a-417b-8303-f371f3523079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:49.938292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 272/544] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:49.938292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99689b4a-d5d2-4d45-bc38-6b3e8ffea048 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:49.938292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99689b4a-d5d2-4d45-bc38-6b3e8ffea048 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:49.938292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99689b4a-d5d2-4d45-bc38-6b3e8ffea048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:49.943352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1ab0d-db9a-417b-8303-f371f3523079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:49.944995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1ab0d-db9a-417b-8303-f371f3523079 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:49.955408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-99689b4a-d5d2-4d45-bc38-6b3e8ffea048 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:49.956767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-99689b4a-d5d2-4d45-bc38-6b3e8ffea048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-c9d1ab0d-db9a-417b-8303-f371f3523079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:49.960901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:49.966225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:49.967189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:49.967189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:49.967189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:49.967189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 273/545] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:49.967189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1ab0d-db9a-417b-8303-f371f3523079 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:50.024774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:50.033948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 273/546] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:50.035355 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791093e-eba9-430e-9190-a3588636e161 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:50.035633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791093e-eba9-430e-9190-a3588636e161 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:50.035900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791093e-eba9-430e-9190-a3588636e161 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:50.037604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 274/547] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:50.038977 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791093e-eba9-430e-9190-a3588636e161 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:50.084104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1ab0d-db9a-417b-8303-f371f3523079 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:50.098511 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:50.127373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16ea59-0369-4c12-913d-30743d8a4da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:50.128038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16ea59-0369-4c12-913d-30743d8a4da7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:50.128433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16ea59-0369-4c12-913d-30743d8a4da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:50.128707 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16ea59-0369-4c12-913d-30743d8a4da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:50.140436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16ea59-0369-4c12-913d-30743d8a4da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:50.142713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16ea59-0369-4c12-913d-30743d8a4da7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-cc16ea59-0369-4c12-913d-30743d8a4da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:50.162786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:50.163742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:50.164628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:50.164628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:50.164628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:50.164628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 274/548] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:50.165969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1e280-25cf-4677-aafb-b19d4a77d379 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:50.165969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1e280-25cf-4677-aafb-b19d4a77d379 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:50.166249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1e280-25cf-4677-aafb-b19d4a77d379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:50.168567 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791093e-eba9-430e-9190-a3588636e161 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:50.193853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-31d1e280-25cf-4677-aafb-b19d4a77d379 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:50.196052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-31d1e280-25cf-4677-aafb-b19d4a77d379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:50.221068 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b69274-478a-4d5d-be72-ddbe650270ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:50.221678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b69274-478a-4d5d-be72-ddbe650270ea None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:50.221956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b69274-478a-4d5d-be72-ddbe650270ea None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:50.222263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b69274-478a-4d5d-be72-ddbe650270ea None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:50.222904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b69274-478a-4d5d-be72-ddbe650270ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:50.223228 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b69274-478a-4d5d-be72-ddbe650270ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:50.224691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b69274-478a-4d5d-be72-ddbe650270ea None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:50.241973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 275/549] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:50.241973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b69274-478a-4d5d-be72-ddbe650270ea None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:50.271843 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 275/550] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:29:50.272948 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1e280-25cf-4677-aafb-b19d4a77d379 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:50.311435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:50.360253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b69274-478a-4d5d-be72-ddbe650270ea None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:50.379726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:50.393201 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1e280-25cf-4677-aafb-b19d4a77d379 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:50.396584 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee82f7-7a1f-4741-a766-ca3c3f4fc4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:50.397261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee82f7-7a1f-4741-a766-ca3c3f4fc4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:50.397545 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee82f7-7a1f-4741-a766-ca3c3f4fc4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:50.397817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee82f7-7a1f-4741-a766-ca3c3f4fc4b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:50.398484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee82f7-7a1f-4741-a766-ca3c3f4fc4b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:50.398788 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee82f7-7a1f-4741-a766-ca3c3f4fc4b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:50.400145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee82f7-7a1f-4741-a766-ca3c3f4fc4b1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:50.413105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 276/551] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:50.418277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee82f7-7a1f-4741-a766-ca3c3f4fc4b1 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:50.430732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bace78a-4a2c-48d8-a4d7-d8d2f9dd9d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:50.431340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bace78a-4a2c-48d8-a4d7-d8d2f9dd9d31 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:50.431613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bace78a-4a2c-48d8-a4d7-d8d2f9dd9d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:50.431877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bace78a-4a2c-48d8-a4d7-d8d2f9dd9d31 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:50.432518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bace78a-4a2c-48d8-a4d7-d8d2f9dd9d31 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:50.433855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bace78a-4a2c-48d8-a4d7-d8d2f9dd9d31 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:50.450227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 276/552] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:29:50 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:50.450924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bace78a-4a2c-48d8-a4d7-d8d2f9dd9d31 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:50.566079 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee82f7-7a1f-4741-a766-ca3c3f4fc4b1 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:50.582714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:50.607175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bace78a-4a2c-48d8-a4d7-d8d2f9dd9d31 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:50.610707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaad83a-4c14-4191-bcb4-54281e65c20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:50.611362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaad83a-4c14-4191-bcb4-54281e65c20e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:50.611639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaad83a-4c14-4191-bcb4-54281e65c20e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:50.611904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaad83a-4c14-4191-bcb4-54281e65c20e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:50.621460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaad83a-4c14-4191-bcb4-54281e65c20e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:50.622925 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaad83a-4c14-4191-bcb4-54281e65c20e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-feaad83a-4c14-4191-bcb4-54281e65c20e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:50.634815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:50.635785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 277/553] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Oct 24 13:29:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:50.637281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaad83a-4c14-4191-bcb4-54281e65c20e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:50.652659 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127aa193-691f-432e-8868-53b48669c10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:50.652975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127aa193-691f-432e-8868-53b48669c10b None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:50.653260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127aa193-691f-432e-8868-53b48669c10b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:50.653529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127aa193-691f-432e-8868-53b48669c10b None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:50.654287 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127aa193-691f-432e-8868-53b48669c10b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:50.655745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127aa193-691f-432e-8868-53b48669c10b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:50.673938 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 277/554] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:29:50 2022] POST /identity/v3/services => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:50.674638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127aa193-691f-432e-8868-53b48669c10b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:50.679206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:50.755283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaad83a-4c14-4191-bcb4-54281e65c20e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:50.799525 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f24cb-da2d-4067-8d5a-863e1722a1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:50.800575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f24cb-da2d-4067-8d5a-863e1722a1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:50.800991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f24cb-da2d-4067-8d5a-863e1722a1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:50.801477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f24cb-da2d-4067-8d5a-863e1722a1b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:50.809976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f24cb-da2d-4067-8d5a-863e1722a1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:50.811834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f24cb-da2d-4067-8d5a-863e1722a1b8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:50.816647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127aa193-691f-432e-8868-53b48669c10b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-fb2f24cb-da2d-4067-8d5a-863e1722a1b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:50.826135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:50.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:50.828184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:50.828184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:50.828184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:50.828184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:50.828184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:50.828184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:50.828184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:50.828184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:50.830271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 278/555] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:50.831278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f24cb-da2d-4067-8d5a-863e1722a1b8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:50.867264 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:50.874742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d02f1-41be-4d56-87ea-8df50279ac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:50.875380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d02f1-41be-4d56-87ea-8df50279ac54 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:50.875653 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d02f1-41be-4d56-87ea-8df50279ac54 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:50.875919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d02f1-41be-4d56-87ea-8df50279ac54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:50.888672 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d02f1-41be-4d56-87ea-8df50279ac54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-d37d02f1-41be-4d56-87ea-8df50279ac54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:50.895010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:50.896824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:50.897742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 278/556] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:29:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:50.902542 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d02f1-41be-4d56-87ea-8df50279ac54 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:50.970531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f24cb-da2d-4067-8d5a-863e1722a1b8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:50.980150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:51.019762 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9597b-89dd-4c69-92b0-581a53e6f86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:51.020753 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9597b-89dd-4c69-92b0-581a53e6f86f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:51.021044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9597b-89dd-4c69-92b0-581a53e6f86f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:51.021319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9597b-89dd-4c69-92b0-581a53e6f86f None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:51.022452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9597b-89dd-4c69-92b0-581a53e6f86f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:51.023200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9597b-89dd-4c69-92b0-581a53e6f86f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:51.024940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9597b-89dd-4c69-92b0-581a53e6f86f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:51.025560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d02f1-41be-4d56-87ea-8df50279ac54 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:51.048665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 279/557] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Oct 24 13:29:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:51.049674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9597b-89dd-4c69-92b0-581a53e6f86f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:51.072070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415050b-b3a2-4b31-8e7d-fca825ac8074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:51.072670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415050b-b3a2-4b31-8e7d-fca825ac8074 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:51.072950 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415050b-b3a2-4b31-8e7d-fca825ac8074 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:51.073217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415050b-b3a2-4b31-8e7d-fca825ac8074 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:51.073962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415050b-b3a2-4b31-8e7d-fca825ac8074 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:51.074299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415050b-b3a2-4b31-8e7d-fca825ac8074 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:51.076241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415050b-b3a2-4b31-8e7d-fca825ac8074 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:51.090878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 279/558] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:51.091724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415050b-b3a2-4b31-8e7d-fca825ac8074 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:51.169303 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:51.184189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9597b-89dd-4c69-92b0-581a53e6f86f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:51.211738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415050b-b3a2-4b31-8e7d-fca825ac8074 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:51.222353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c192c4-da57-4710-9119-74a354244f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:51.223354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c192c4-da57-4710-9119-74a354244f09 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:51.224450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c192c4-da57-4710-9119-74a354244f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:51.224929 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c192c4-da57-4710-9119-74a354244f09 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:51.225796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c192c4-da57-4710-9119-74a354244f09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:51.226427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c192c4-da57-4710-9119-74a354244f09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:51.228043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c192c4-da57-4710-9119-74a354244f09 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:51.237225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 280/559] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:51.238344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c192c4-da57-4710-9119-74a354244f09 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:51.242455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:51.255380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ea686-c635-4e2e-91df-1affb2f905e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:51.256095 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ea686-c635-4e2e-91df-1affb2f905e6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:51.256374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ea686-c635-4e2e-91df-1affb2f905e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:51.256636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ea686-c635-4e2e-91df-1affb2f905e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:51.331711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ea686-c635-4e2e-91df-1affb2f905e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:51.333890 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ea686-c635-4e2e-91df-1affb2f905e6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:51.351456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c192c4-da57-4710-9119-74a354244f09 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:51.392250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50f013-ae5f-4d4a-99cd-3633119ee5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:51.393364 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50f013-ae5f-4d4a-99cd-3633119ee5cd None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:51.393918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50f013-ae5f-4d4a-99cd-3633119ee5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:51.394518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50f013-ae5f-4d4a-99cd-3633119ee5cd None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/cb6d3abe410746828fb07621ef1a1a5c/roles/037adbca67514eda861c68218a60c50a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:51.432095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50f013-ae5f-4d4a-99cd-3633119ee5cd None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=cb6d3abe410746828fb07621ef1a1a5c, role_id=037adbca67514eda861c68218a60c50a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:51.433546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50f013-ae5f-4d4a-99cd-3633119ee5cd None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:51.447182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:51.448857 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 280/560] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:29:51.450053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ea686-c635-4e2e-91df-1affb2f905e6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:51.457850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 281/561] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:29:51 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/cb6d3abe410746828fb07621ef1a1a5c/roles/037adbca67514eda861c68218a60c50a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:51.459431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50f013-ae5f-4d4a-99cd-3633119ee5cd None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:51.483888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:51.647392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ea686-c635-4e2e-91df-1affb2f905e6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:51.662285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50f013-ae5f-4d4a-99cd-3633119ee5cd None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:51.697157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5870ee-44ab-4f46-86d3-f935f747cba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:51.698746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5870ee-44ab-4f46-86d3-f935f747cba3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:51.699508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5870ee-44ab-4f46-86d3-f935f747cba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:51.699968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5870ee-44ab-4f46-86d3-f935f747cba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:51.707044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94867cc-c9e9-4467-a64d-a1c14c089b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:51.707781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94867cc-c9e9-4467-a64d-a1c14c089b02 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:51.708976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94867cc-c9e9-4467-a64d-a1c14c089b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:51.708976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94867cc-c9e9-4467-a64d-a1c14c089b02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:51.713597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5870ee-44ab-4f46-86d3-f935f747cba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:51.715284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5870ee-44ab-4f46-86d3-f935f747cba3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:51.721194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94867cc-c9e9-4467-a64d-a1c14c089b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:51.722741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94867cc-c9e9-4467-a64d-a1c14c089b02 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-3d5870ee-44ab-4f46-86d3-f935f747cba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:51.734030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:51.735036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:51.735910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:51.735910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:51.735910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:51.737753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 281/562] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f94867cc-c9e9-4467-a64d-a1c14c089b02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:51.740510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:51.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:51.742673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:51.742673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:51.742673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:51.746066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5870ee-44ab-4f46-86d3-f935f747cba3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:51.748871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 282/563] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:51.749758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94867cc-c9e9-4467-a64d-a1c14c089b02 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:51.825649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:51.842789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:51.869338 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5870ee-44ab-4f46-86d3-f935f747cba3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:51.905211 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6961f6d-9e99-4f97-bd8e-d0bb294a2715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:51.906207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961f6d-9e99-4f97-bd8e-d0bb294a2715 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:51.906684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961f6d-9e99-4f97-bd8e-d0bb294a2715 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:51.907146 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961f6d-9e99-4f97-bd8e-d0bb294a2715 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:51.907972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6961f6d-9e99-4f97-bd8e-d0bb294a2715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:51.908461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6961f6d-9e99-4f97-bd8e-d0bb294a2715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:51.910029 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6961f6d-9e99-4f97-bd8e-d0bb294a2715 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:51.925194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 282/564] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:51.931408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6961f6d-9e99-4f97-bd8e-d0bb294a2715 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:51.937091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94867cc-c9e9-4467-a64d-a1c14c089b02 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:52.006878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643048e8-c9b4-4ebf-ac89-bbe1c07716da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:52.007505 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643048e8-c9b4-4ebf-ac89-bbe1c07716da None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:52.007789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643048e8-c9b4-4ebf-ac89-bbe1c07716da None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:52.008064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643048e8-c9b4-4ebf-ac89-bbe1c07716da None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:52.008706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643048e8-c9b4-4ebf-ac89-bbe1c07716da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:52.009008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643048e8-c9b4-4ebf-ac89-bbe1c07716da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:52.010400 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643048e8-c9b4-4ebf-ac89-bbe1c07716da None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:52.031835 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 283/565] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:52.037600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643048e8-c9b4-4ebf-ac89-bbe1c07716da None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:52.040264 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6961f6d-9e99-4f97-bd8e-d0bb294a2715 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:52.080426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a7516-1c97-473e-81c8-925d9741703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:52.081448 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a7516-1c97-473e-81c8-925d9741703d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:52.081925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a7516-1c97-473e-81c8-925d9741703d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:52.082475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a7516-1c97-473e-81c8-925d9741703d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:52.093061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a7516-1c97-473e-81c8-925d9741703d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:52.094935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a7516-1c97-473e-81c8-925d9741703d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:52.102610 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-935a7516-1c97-473e-81c8-925d9741703d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:52.111439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:52.112436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:52.113374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:52.113374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:52.113374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:52.113374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:52.113374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:52.113374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:52.113374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:52.113374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:52.113374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:29:52.113374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:52.116010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 283/566] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:29:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:52.121968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a7516-1c97-473e-81c8-925d9741703d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:52.238632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a7516-1c97-473e-81c8-925d9741703d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:52.252673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643048e8-c9b4-4ebf-ac89-bbe1c07716da None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:52.267556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:52.300935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e52ce2-9fa6-4f82-8702-726e7dc7f70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:52.301546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e52ce2-9fa6-4f82-8702-726e7dc7f70a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:52.302058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e52ce2-9fa6-4f82-8702-726e7dc7f70a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:52.302150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e52ce2-9fa6-4f82-8702-726e7dc7f70a None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:52.302858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e52ce2-9fa6-4f82-8702-726e7dc7f70a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:52.303160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e52ce2-9fa6-4f82-8702-726e7dc7f70a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:52.304603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e52ce2-9fa6-4f82-8702-726e7dc7f70a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:52.309945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526e45d-0255-443f-8c7e-9e0864411854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:52.310605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526e45d-0255-443f-8c7e-9e0864411854 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:52.310884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526e45d-0255-443f-8c7e-9e0864411854 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:52.311150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526e45d-0255-443f-8c7e-9e0864411854 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:52.322953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526e45d-0255-443f-8c7e-9e0864411854 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:52.324393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526e45d-0255-443f-8c7e-9e0864411854 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:52.339970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 284/567] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Oct 24 13:29:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b526e45d-0255-443f-8c7e-9e0864411854 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:52.345743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:52.346651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:52.347602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:52.347602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:52.347602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:52.347602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:52.347602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:52.347602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:52.347602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:52.347602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:52.348068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e52ce2-9fa6-4f82-8702-726e7dc7f70a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:52.350951 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 284/568] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:52.354606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526e45d-0255-443f-8c7e-9e0864411854 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:52.446527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:52.524227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e52ce2-9fa6-4f82-8702-726e7dc7f70a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:52.528775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526e45d-0255-443f-8c7e-9e0864411854 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:52.585851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912549bc-93d4-4567-87d2-87336659cefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:52.586522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912549bc-93d4-4567-87d2-87336659cefe None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:52.586802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912549bc-93d4-4567-87d2-87336659cefe None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:52.587067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912549bc-93d4-4567-87d2-87336659cefe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:52.594232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca5aab-f180-4a00-9407-65b93fc6584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:52.594842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca5aab-f180-4a00-9407-65b93fc6584d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:52.595119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca5aab-f180-4a00-9407-65b93fc6584d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:52.595389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca5aab-f180-4a00-9407-65b93fc6584d None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:52.596073 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca5aab-f180-4a00-9407-65b93fc6584d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:52.596372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca5aab-f180-4a00-9407-65b93fc6584d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:52.597784 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca5aab-f180-4a00-9407-65b93fc6584d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:52.612390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912549bc-93d4-4567-87d2-87336659cefe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:52.613837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912549bc-93d4-4567-87d2-87336659cefe None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:52.618255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 285/569] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:52.623987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca5aab-f180-4a00-9407-65b93fc6584d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-912549bc-93d4-4567-87d2-87336659cefe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:52.629592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:52.631228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:52.632100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:52.632100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:52.632100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:52.635874 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 285/570] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:52.637087 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912549bc-93d4-4567-87d2-87336659cefe None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:52.696927 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:52.771230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912549bc-93d4-4567-87d2-87336659cefe None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:52.782632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:52.829351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca5aab-f180-4a00-9407-65b93fc6584d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:52.832650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f4f6ef-2640-403f-acf7-e50a7485a267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:52.834059 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4f6ef-2640-403f-acf7-e50a7485a267 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:52.834502 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4f6ef-2640-403f-acf7-e50a7485a267 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:52.834778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4f6ef-2640-403f-acf7-e50a7485a267 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:52.835475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f4f6ef-2640-403f-acf7-e50a7485a267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:52.835777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f4f6ef-2640-403f-acf7-e50a7485a267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:52.837725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f4f6ef-2640-403f-acf7-e50a7485a267 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:52.852279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 286/571] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:52.853738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203f5ff-0a36-474c-afff-e568b7da995b None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:52.854019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203f5ff-0a36-474c-afff-e568b7da995b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:52.854424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203f5ff-0a36-474c-afff-e568b7da995b None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:52.855625 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 287/572] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:52.860355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203f5ff-0a36-474c-afff-e568b7da995b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:52.867571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431d679-8987-4a23-947c-6e738499ce83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:52.868468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431d679-8987-4a23-947c-6e738499ce83 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:52.868931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431d679-8987-4a23-947c-6e738499ce83 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:52.869406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431d679-8987-4a23-947c-6e738499ce83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:52.870466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431d679-8987-4a23-947c-6e738499ce83 None admin] RBAC: Adding query filter params (role.id=8b32d66a3cb3483c8ede361ad8637f66, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=18bb93669846435a85eb34771cfb09fe) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:52.870993 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431d679-8987-4a23-947c-6e738499ce83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:52.872698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431d679-8987-4a23-947c-6e738499ce83 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:52.880363 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 286/573] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:29:52 2022] GET /identity/v3/role_assignments?user.id=18bb93669846435a85eb34771cfb09fe&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=8b32d66a3cb3483c8ede361ad8637f66 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:52.881867 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079911a-8ef0-434f-b19f-ad9bd0a5ef57 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:52.882361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079911a-8ef0-434f-b19f-ad9bd0a5ef57 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:52.882817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079911a-8ef0-434f-b19f-ad9bd0a5ef57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:52.917154 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0079911a-8ef0-434f-b19f-ad9bd0a5ef57 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:52.919655 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0079911a-8ef0-434f-b19f-ad9bd0a5ef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:53.065998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203f5ff-0a36-474c-afff-e568b7da995b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:53.070286 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 287/574] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:29:53.112162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c48f6-d6e5-4da8-b901-d537184ad2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:53.113175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c48f6-d6e5-4da8-b901-d537184ad2bd None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:53.113679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c48f6-d6e5-4da8-b901-d537184ad2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:53.114335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c48f6-d6e5-4da8-b901-d537184ad2bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:53.123549 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c48f6-d6e5-4da8-b901-d537184ad2bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:53.125291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c48f6-d6e5-4da8-b901-d537184ad2bd None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-f91c48f6-d6e5-4da8-b901-d537184ad2bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:53.138810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:53.139755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:53.140616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:53.140616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:53.140616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:53.140616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:53.140616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:53.140616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:53.140616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:53.140616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:53.142978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 288/575] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:53.150599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079911a-8ef0-434f-b19f-ad9bd0a5ef57 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:53.188920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:53.219818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4e73e-c0ea-49cf-b303-4e680ca08910 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:53.220016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4e73e-c0ea-49cf-b303-4e680ca08910 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:53.220510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4e73e-c0ea-49cf-b303-4e680ca08910 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:53.221709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 289/576] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:53.227150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120fec7-853a-4b3e-8c49-33a84e4744fe None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:53.227511 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120fec7-853a-4b3e-8c49-33a84e4744fe None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:53.227899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120fec7-853a-4b3e-8c49-33a84e4744fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:53.248092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9120fec7-853a-4b3e-8c49-33a84e4744fe None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:53.249409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9120fec7-853a-4b3e-8c49-33a84e4744fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:53.279270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:53.292701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079911a-8ef0-434f-b19f-ad9bd0a5ef57 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:53.348259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07fbd2-13de-47d7-aae4-4ee89749b573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:53.349524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07fbd2-13de-47d7-aae4-4ee89749b573 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:53.349984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07fbd2-13de-47d7-aae4-4ee89749b573 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:53.350291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07fbd2-13de-47d7-aae4-4ee89749b573 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:53.350996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07fbd2-13de-47d7-aae4-4ee89749b573 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:53.351300 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07fbd2-13de-47d7-aae4-4ee89749b573 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:53.352934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07fbd2-13de-47d7-aae4-4ee89749b573 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:53.365574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 288/577] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:53.371213 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07fbd2-13de-47d7-aae4-4ee89749b573 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:53.391396 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 290/578] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:29:53.392237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120fec7-853a-4b3e-8c49-33a84e4744fe None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:53.473037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:53.506321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07fbd2-13de-47d7-aae4-4ee89749b573 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:53.546410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120fec7-853a-4b3e-8c49-33a84e4744fe None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:53.569016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8680d75-4eb0-421b-858e-3acda5588bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:53.570034 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8680d75-4eb0-421b-858e-3acda5588bad None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:53.570548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8680d75-4eb0-421b-858e-3acda5588bad None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:53.571031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8680d75-4eb0-421b-858e-3acda5588bad None admin] PATH_INFO: `/v3/services/network` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:53.571818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8680d75-4eb0-421b-858e-3acda5588bad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:53.574759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8680d75-4eb0-421b-858e-3acda5588bad None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-d8680d75-4eb0-421b-858e-3acda5588bad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:53.591454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:53.592579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:29:53.593511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:53.598357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 289/579] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:53 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:53.607235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8680d75-4eb0-421b-858e-3acda5588bad None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:53.610476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907c113-7125-4972-beef-e6c532fe52d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:53.611427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c113-7125-4972-beef-e6c532fe52d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:53.611938 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c113-7125-4972-beef-e6c532fe52d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:53.612415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907c113-7125-4972-beef-e6c532fe52d2 None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/0ea0d300a51c41fba4d4193dd2251a50/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:53.615282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907c113-7125-4972-beef-e6c532fe52d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=0ea0d300a51c41fba4d4193dd2251a50, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:53.617048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907c113-7125-4972-beef-e6c532fe52d2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:53.641853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 291/580] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:29:53 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/0ea0d300a51c41fba4d4193dd2251a50/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:53.748805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907c113-7125-4972-beef-e6c532fe52d2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:53.770208 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:53.778511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8680d75-4eb0-421b-858e-3acda5588bad None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:53.859440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59291847-9579-461c-abc1-6a3f00e8cb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:53.860195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59291847-9579-461c-abc1-6a3f00e8cb5e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:53.860478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59291847-9579-461c-abc1-6a3f00e8cb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:53.860749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59291847-9579-461c-abc1-6a3f00e8cb5e None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:53.861384 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59291847-9579-461c-abc1-6a3f00e8cb5e None admin] RBAC: Adding query filter params (name=network) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:53.861684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59291847-9579-461c-abc1-6a3f00e8cb5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:53.863346 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59291847-9579-461c-abc1-6a3f00e8cb5e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:53.868732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 290/581] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:29:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:53.874410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59291847-9579-461c-abc1-6a3f00e8cb5e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:53.892715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:53.902643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907c113-7125-4972-beef-e6c532fe52d2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:53.940400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b772c-e866-489e-a6fe-dfa80cbfb7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:53.941126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b772c-e866-489e-a6fe-dfa80cbfb7fe None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:53.941429 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b772c-e866-489e-a6fe-dfa80cbfb7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:53.941762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b772c-e866-489e-a6fe-dfa80cbfb7fe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:53.942927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b772c-e866-489e-a6fe-dfa80cbfb7fe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:53.944284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b772c-e866-489e-a6fe-dfa80cbfb7fe None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-3b6b772c-e866-489e-a6fe-dfa80cbfb7fe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:53.955720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:53.956796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:53.959301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 292/582] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Oct 24 13:29:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:53.974307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b772c-e866-489e-a6fe-dfa80cbfb7fe None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:53.995755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59291847-9579-461c-abc1-6a3f00e8cb5e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:54.031858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7126c12-ab66-48f8-907a-e52447aea0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:54.032475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7126c12-ab66-48f8-907a-e52447aea0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:54.032755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7126c12-ab66-48f8-907a-e52447aea0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:54.033028 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7126c12-ab66-48f8-907a-e52447aea0b3 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:54.033658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7126c12-ab66-48f8-907a-e52447aea0b3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:54.034069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7126c12-ab66-48f8-907a-e52447aea0b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:54.035480 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7126c12-ab66-48f8-907a-e52447aea0b3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:54.042468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 291/583] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:29:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:54.048008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7126c12-ab66-48f8-907a-e52447aea0b3 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:54.138946 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:54.158762 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7126c12-ab66-48f8-907a-e52447aea0b3 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:54.160283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:54.197374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00ccc1-e4d0-4320-ab7d-3aa3e4606e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:54.198456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00ccc1-e4d0-4320-ab7d-3aa3e4606e6e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:54.198964 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00ccc1-e4d0-4320-ab7d-3aa3e4606e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:54.199466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00ccc1-e4d0-4320-ab7d-3aa3e4606e6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:54.200384 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00ccc1-e4d0-4320-ab7d-3aa3e4606e6e None admin] RBAC: Adding query filter params (interface=public, service_id=bfc3bb2ba109490bb5206f34c842349d, region_id=RegionOne) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:54.200907 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00ccc1-e4d0-4320-ab7d-3aa3e4606e6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:54.203001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00ccc1-e4d0-4320-ab7d-3aa3e4606e6e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:54.213311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 292/584] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Oct 24 13:29:54 2022] GET /identity/v3/endpoints?service_id=bfc3bb2ba109490bb5206f34c842349d&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:54.220299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00ccc1-e4d0-4320-ab7d-3aa3e4606e6e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:54.236814 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b772c-e866-489e-a6fe-dfa80cbfb7fe None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:54.314809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977934e2-36c8-4668-9a44-1851b45e80f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:54.315382 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977934e2-36c8-4668-9a44-1851b45e80f0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:54.315661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977934e2-36c8-4668-9a44-1851b45e80f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:54.315940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977934e2-36c8-4668-9a44-1851b45e80f0 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:54.316821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977934e2-36c8-4668-9a44-1851b45e80f0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:54.317011 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977934e2-36c8-4668-9a44-1851b45e80f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:54.322619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977934e2-36c8-4668-9a44-1851b45e80f0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:54.332687 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 293/585] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Oct 24 13:29:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:54.342097 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cafd9b-e7a4-45df-a4e8-65e9bc9ed804 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:54.342434 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cafd9b-e7a4-45df-a4e8-65e9bc9ed804 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:54.342707 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cafd9b-e7a4-45df-a4e8-65e9bc9ed804 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:54.344240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 294/586] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:29:54.349361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727f8a2-c479-4dab-8241-9ee80395f289 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:54.349361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727f8a2-c479-4dab-8241-9ee80395f289 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:54.349686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727f8a2-c479-4dab-8241-9ee80395f289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:54.366867 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00ccc1-e4d0-4320-ab7d-3aa3e4606e6e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:54.367481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8727f8a2-c479-4dab-8241-9ee80395f289 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:54.368613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8727f8a2-c479-4dab-8241-9ee80395f289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:54.411673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0021b93-455a-4e2f-871e-9f4d0060d78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:54.413012 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0021b93-455a-4e2f-871e-9f4d0060d78e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:54.413830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0021b93-455a-4e2f-871e-9f4d0060d78e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:54.414487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0021b93-455a-4e2f-871e-9f4d0060d78e None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:54.415405 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0021b93-455a-4e2f-871e-9f4d0060d78e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:54.417787 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0021b93-455a-4e2f-871e-9f4d0060d78e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:54.425023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 293/587] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Oct 24 13:29:54 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:54.426102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0021b93-455a-4e2f-871e-9f4d0060d78e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:54.436253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:54.467687 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 295/588] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:29:54.539603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0021b93-455a-4e2f-871e-9f4d0060d78e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:54.577458 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b9649-2f50-4034-a236-8d9c3bc14c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:54.578458 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b9649-2f50-4034-a236-8d9c3bc14c6c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:54.578939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b9649-2f50-4034-a236-8d9c3bc14c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:54.579393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b9649-2f50-4034-a236-8d9c3bc14c6c None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:54.580234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b9649-2f50-4034-a236-8d9c3bc14c6c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:54.580728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b9649-2f50-4034-a236-8d9c3bc14c6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:54.582338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b9649-2f50-4034-a236-8d9c3bc14c6c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:54.589340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 294/589] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Oct 24 13:29:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:54.595587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727f8a2-c479-4dab-8241-9ee80395f289 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:54.679309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:54.708558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727f8a2-c479-4dab-8241-9ee80395f289 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:54.745731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ba063-4b16-4150-a7a8-5981fa78a5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:54.746700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ba063-4b16-4150-a7a8-5981fa78a5ce None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:54.747166 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ba063-4b16-4150-a7a8-5981fa78a5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:54.747649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ba063-4b16-4150-a7a8-5981fa78a5ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:54.748539 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ba063-4b16-4150-a7a8-5981fa78a5ce None admin] RBAC: Adding query filter params (interface=public, service_id=669e59fa22ac4e7c80729a6abe658589, region_id=RegionOne) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:54.749024 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ba063-4b16-4150-a7a8-5981fa78a5ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:54.750599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ba063-4b16-4150-a7a8-5981fa78a5ce None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:54.758863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 296/590] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Oct 24 13:29:54 2022] GET /identity/v3/endpoints?service_id=669e59fa22ac4e7c80729a6abe658589&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:54.764627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b9649-2f50-4034-a236-8d9c3bc14c6c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:54.843890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:54.877588 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b9649-2f50-4034-a236-8d9c3bc14c6c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:54.900907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ba063-4b16-4150-a7a8-5981fa78a5ce None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:54.916245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332a31c-4cc2-47ad-9066-378992c52836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:54.916864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332a31c-4cc2-47ad-9066-378992c52836 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:54.917330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332a31c-4cc2-47ad-9066-378992c52836 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:54.917464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332a31c-4cc2-47ad-9066-378992c52836 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:54.917987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332a31c-4cc2-47ad-9066-378992c52836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:54.919528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332a31c-4cc2-47ad-9066-378992c52836 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:54.926674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 295/591] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Oct 24 13:29:54 2022] GET /identity/v3/services => generated 772 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:55.028358 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ba063-4b16-4150-a7a8-5981fa78a5ce None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:55.070349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ef4fb-1715-4a3d-85f9-e97557dd2a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:55.071480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ef4fb-1715-4a3d-85f9-e97557dd2a57 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:55.071973 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ef4fb-1715-4a3d-85f9-e97557dd2a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:55.072455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ef4fb-1715-4a3d-85f9-e97557dd2a57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:55.081501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:55.088578 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ef4fb-1715-4a3d-85f9-e97557dd2a57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-a25ef4fb-1715-4a3d-85f9-e97557dd2a57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:55.094042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:55.095114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:29:55.095962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:55.099297 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 297/592] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:29:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:55.105944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332a31c-4cc2-47ad-9066-378992c52836 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:55.215436 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332a31c-4cc2-47ad-9066-378992c52836 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:55.253190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7b30e-be7c-42b7-b669-d46f3bcaad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:55.253807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b30e-be7c-42b7-b669-d46f3bcaad24 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:55.254083 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b30e-be7c-42b7-b669-d46f3bcaad24 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:55.254380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7b30e-be7c-42b7-b669-d46f3bcaad24 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:55.255076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7b30e-be7c-42b7-b669-d46f3bcaad24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:55.255377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7b30e-be7c-42b7-b669-d46f3bcaad24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:55.256936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7b30e-be7c-42b7-b669-d46f3bcaad24 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:55.258393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:55.268923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 296/593] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:29:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:55.271979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ef4fb-1715-4a3d-85f9-e97557dd2a57 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:55.377520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ef4fb-1715-4a3d-85f9-e97557dd2a57 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:55.413410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d0865-30af-43f9-98a7-ff4aefa96674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:55.414047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d0865-30af-43f9-98a7-ff4aefa96674 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:55.414352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d0865-30af-43f9-98a7-ff4aefa96674 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:55.414622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d0865-30af-43f9-98a7-ff4aefa96674 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:55.429631 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d0865-30af-43f9-98a7-ff4aefa96674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:55.431171 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d0865-30af-43f9-98a7-ff4aefa96674 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-d35d0865-30af-43f9-98a7-ff4aefa96674 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:55.446105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:55.447149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:55.448680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:55.448680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:55.448680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:55.448680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 298/594] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:55.453055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7b30e-be7c-42b7-b669-d46f3bcaad24 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:55.576717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7b30e-be7c-42b7-b669-d46f3bcaad24 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:55.619095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd02c83-a43c-4ec5-9bb0-b1737a5b1447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:55.619712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd02c83-a43c-4ec5-9bb0-b1737a5b1447 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:55.619985 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd02c83-a43c-4ec5-9bb0-b1737a5b1447 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:55.620249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd02c83-a43c-4ec5-9bb0-b1737a5b1447 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:55.620885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd02c83-a43c-4ec5-9bb0-b1737a5b1447 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:55.621181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd02c83-a43c-4ec5-9bb0-b1737a5b1447 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:55.622553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd02c83-a43c-4ec5-9bb0-b1737a5b1447 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:55.641267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 297/595] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:55.647138 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d0865-30af-43f9-98a7-ff4aefa96674 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:55.666278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a90372-a5e2-4971-9561-2822c661328e None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:55.666574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a90372-a5e2-4971-9561-2822c661328e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:55.666829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a90372-a5e2-4971-9561-2822c661328e None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:55.668300 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 298/596] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:55.675099 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d7312-17db-4215-881c-2bea18053400 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:55.675369 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d7312-17db-4215-881c-2bea18053400 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:55.675644 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d7312-17db-4215-881c-2bea18053400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:55.703157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5e5d7312-17db-4215-881c-2bea18053400 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:55.704327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5e5d7312-17db-4215-881c-2bea18053400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:55.714076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:55.787131 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d0865-30af-43f9-98a7-ff4aefa96674 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:55.800910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:55.829368 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed716-16ee-48a8-8093-8cae4f783758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:55.830446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed716-16ee-48a8-8093-8cae4f783758 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:55.830893 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed716-16ee-48a8-8093-8cae4f783758 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:55.831160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed716-16ee-48a8-8093-8cae4f783758 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:55.836027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 299/597] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:29:55.840986 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fed716-16ee-48a8-8093-8cae4f783758 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:55.842470 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fed716-16ee-48a8-8093-8cae4f783758 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-d2fed716-16ee-48a8-8093-8cae4f783758 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:55.863756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:55.865847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:55.866815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:55.866815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:55.866815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:55.866815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:55.866815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:55.866815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:55.866815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:55.866815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:55.866815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:29:55.866815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:55.869356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 299/598] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Oct 24 13:29:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:55.874796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d7312-17db-4215-881c-2bea18053400 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:56.053583 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d7312-17db-4215-881c-2bea18053400 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:56.101116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a1eb-5a6e-49f3-a7d7-b45252140a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:56.101737 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a1eb-5a6e-49f3-a7d7-b45252140a69 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:56.102020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a1eb-5a6e-49f3-a7d7-b45252140a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:56.102320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a1eb-5a6e-49f3-a7d7-b45252140a69 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:56.103122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346a1eb-5a6e-49f3-a7d7-b45252140a69 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:56.103429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346a1eb-5a6e-49f3-a7d7-b45252140a69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:56.104867 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346a1eb-5a6e-49f3-a7d7-b45252140a69 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:56.114875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed716-16ee-48a8-8093-8cae4f783758 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:56.133514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 300/599] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Oct 24 13:29:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:56.143688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a1eb-5a6e-49f3-a7d7-b45252140a69 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:56.229181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:56.322332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed716-16ee-48a8-8093-8cae4f783758 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:56.346735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:56.347076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a1eb-5a6e-49f3-a7d7-b45252140a69 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:56.406546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca48c9e-186f-449d-8dcf-714635a1f2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:56.408073 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca48c9e-186f-449d-8dcf-714635a1f2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:56.408921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efebed-0b4f-49e9-8894-5ed5feb68ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:56.409433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca48c9e-186f-449d-8dcf-714635a1f2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:56.409924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca48c9e-186f-449d-8dcf-714635a1f2d6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:56.410768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efebed-0b4f-49e9-8894-5ed5feb68ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:56.412026 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca48c9e-186f-449d-8dcf-714635a1f2d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:56.412729 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efebed-0b4f-49e9-8894-5ed5feb68ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:56.413206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efebed-0b4f-49e9-8894-5ed5feb68ce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:56.414870 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca48c9e-186f-449d-8dcf-714635a1f2d6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-8ca48c9e-186f-449d-8dcf-714635a1f2d6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:56.433594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:56.434629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:29:56.435499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:56.438749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 300/600] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Oct 24 13:29:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:56.447234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca48c9e-186f-449d-8dcf-714635a1f2d6 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:56.524052 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efebed-0b4f-49e9-8894-5ed5feb68ce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:56.526258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efebed-0b4f-49e9-8894-5ed5feb68ce3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:56.579230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca48c9e-186f-449d-8dcf-714635a1f2d6 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:56.595175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:56.632675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17ddeb-27b2-4284-9d3e-07ae27c81974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:56.633673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17ddeb-27b2-4284-9d3e-07ae27c81974 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:56.634192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17ddeb-27b2-4284-9d3e-07ae27c81974 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:56.634693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17ddeb-27b2-4284-9d3e-07ae27c81974 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:56.635550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17ddeb-27b2-4284-9d3e-07ae27c81974 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:56.636069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17ddeb-27b2-4284-9d3e-07ae27c81974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:56.637752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17ddeb-27b2-4284-9d3e-07ae27c81974 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:56.643569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 301/601] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Oct 24 13:29:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:56.645082 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd44a00-bc82-4b49-b9ef-8abcbb110c87 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:56.646349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd44a00-bc82-4b49-b9ef-8abcbb110c87 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:56.646959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd44a00-bc82-4b49-b9ef-8abcbb110c87 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:56.649022 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 302/602] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:56.651058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd44a00-bc82-4b49-b9ef-8abcbb110c87 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:56.674267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 301/603] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 24 13:29:56.676059 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bcec6-8a35-4d4a-85fb-678c9d61a44d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:56.676200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bcec6-8a35-4d4a-85fb-678c9d61a44d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:56.676388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bcec6-8a35-4d4a-85fb-678c9d61a44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:56.698615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-565bcec6-8a35-4d4a-85fb-678c9d61a44d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:56.700004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-565bcec6-8a35-4d4a-85fb-678c9d61a44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:56.711560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:56.786512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 302/604] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:29:56.787706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565bcec6-8a35-4d4a-85fb-678c9d61a44d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:56.788972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd44a00-bc82-4b49-b9ef-8abcbb110c87 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:56.835784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56987da-c6f1-46a2-bb22-e8e6af8eeee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:56.837358 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56987da-c6f1-46a2-bb22-e8e6af8eeee7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:56.837885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56987da-c6f1-46a2-bb22-e8e6af8eeee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:56.838407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56987da-c6f1-46a2-bb22-e8e6af8eeee7 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:56.839463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56987da-c6f1-46a2-bb22-e8e6af8eeee7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:56.839973 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56987da-c6f1-46a2-bb22-e8e6af8eeee7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:56.841713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56987da-c6f1-46a2-bb22-e8e6af8eeee7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:56.847928 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 303/605] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Oct 24 13:29:56 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:56.966992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:57.008622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565bcec6-8a35-4d4a-85fb-678c9d61a44d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:57.048608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a601517-856e-4547-ad82-7c4c8f4a31ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:57.049261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a601517-856e-4547-ad82-7c4c8f4a31ec None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.049542 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a601517-856e-4547-ad82-7c4c8f4a31ec None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:57.049814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a601517-856e-4547-ad82-7c4c8f4a31ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:57.062444 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a601517-856e-4547-ad82-7c4c8f4a31ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:57.064241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a601517-856e-4547-ad82-7c4c8f4a31ec None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-4a601517-856e-4547-ad82-7c4c8f4a31ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:57.087757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:57.088745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:57.089633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:57.089633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:57.089633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:57.089633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 303/606] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:57.094846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56987da-c6f1-46a2-bb22-e8e6af8eeee7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:57.206841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a601517-856e-4547-ad82-7c4c8f4a31ec None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:57.210708 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56987da-c6f1-46a2-bb22-e8e6af8eeee7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:57.214480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:57.300118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e76f0-e127-4b3f-b526-8d8c4568e9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:57.301168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e76f0-e127-4b3f-b526-8d8c4568e9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.301694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e76f0-e127-4b3f-b526-8d8c4568e9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:57.302197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e76f0-e127-4b3f-b526-8d8c4568e9d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:57.303213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e76f0-e127-4b3f-b526-8d8c4568e9d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:57.303718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e76f0-e127-4b3f-b526-8d8c4568e9d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:57.306381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e76f0-e127-4b3f-b526-8d8c4568e9d3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:57.332644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 304/607] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:57.339187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e76f0-e127-4b3f-b526-8d8c4568e9d3 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:57.456905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e76f0-e127-4b3f-b526-8d8c4568e9d3 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:57.468343 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a601517-856e-4547-ad82-7c4c8f4a31ec None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:57.494737 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f789d87-d1d7-44cb-a997-ea4a78b9704b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:57.495724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f789d87-d1d7-44cb-a997-ea4a78b9704b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.496230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f789d87-d1d7-44cb-a997-ea4a78b9704b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:57.496700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f789d87-d1d7-44cb-a997-ea4a78b9704b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:57.505435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f789d87-d1d7-44cb-a997-ea4a78b9704b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:57.507172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f789d87-d1d7-44cb-a997-ea4a78b9704b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6f789d87-d1d7-44cb-a997-ea4a78b9704b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:57.519485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:57.521004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:57.522021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:57.522021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:57.522021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:57.522021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:57.522021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:57.522021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:57.522021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:57.522021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:57.524445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 305/608] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:57.526534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:57.535691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f789d87-d1d7-44cb-a997-ea4a78b9704b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:57.551779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a13a0-bcaf-4097-881c-26da0b7dad36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:57.552418 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a13a0-bcaf-4097-881c-26da0b7dad36 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.552767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a13a0-bcaf-4097-881c-26da0b7dad36 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:57.553033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a13a0-bcaf-4097-881c-26da0b7dad36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:57.565466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a13a0-bcaf-4097-881c-26da0b7dad36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:29:57.570451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-8e5a13a0-bcaf-4097-881c-26da0b7dad36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:29:57.573465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:57.574579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:29:57.575476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:57.578208 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 304/609] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:29:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:29:57.579580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c6466-787c-42d8-bba3-08fc139f8b47 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.580096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c6466-787c-42d8-bba3-08fc139f8b47 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:57.580617 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c6466-787c-42d8-bba3-08fc139f8b47 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:57.586949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 305/610] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:57.587970 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c6466-787c-42d8-bba3-08fc139f8b47 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:57.657771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f789d87-d1d7-44cb-a997-ea4a78b9704b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:57.709270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607e153-2fce-4185-9b23-9afc79090b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:57.710471 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607e153-2fce-4185-9b23-9afc79090b90 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.710823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607e153-2fce-4185-9b23-9afc79090b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:57.711168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607e153-2fce-4185-9b23-9afc79090b90 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:57.711953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607e153-2fce-4185-9b23-9afc79090b90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:57.712331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607e153-2fce-4185-9b23-9afc79090b90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:57.713955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607e153-2fce-4185-9b23-9afc79090b90 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:57.723640 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c6466-787c-42d8-bba3-08fc139f8b47 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:57.728237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 306/611] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:57.729547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a46d2-55ce-4384-aae3-076a71660d2e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.729902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a46d2-55ce-4384-aae3-076a71660d2e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:57.730287 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a46d2-55ce-4384-aae3-076a71660d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:57.750886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8f3a46d2-55ce-4384-aae3-076a71660d2e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:57.752183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8f3a46d2-55ce-4384-aae3-076a71660d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:57.766780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2fb42-fc6b-464b-a870-16489fa601a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:57.767958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2fb42-fc6b-464b-a870-16489fa601a2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.768418 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2fb42-fc6b-464b-a870-16489fa601a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:57.769477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2fb42-fc6b-464b-a870-16489fa601a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:57.770688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2fb42-fc6b-464b-a870-16489fa601a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:57.771191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2fb42-fc6b-464b-a870-16489fa601a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:57.773429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2fb42-fc6b-464b-a870-16489fa601a2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:57.785741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 306/612] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:29:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:57.787225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2fb42-fc6b-464b-a870-16489fa601a2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:57.829521 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 307/613] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:29:57.830694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a46d2-55ce-4384-aae3-076a71660d2e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:57.842797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:57.857014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:57.906798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2fb42-fc6b-464b-a870-16489fa601a2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:57.944813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d74556-db3e-42f0-b774-64d3f1121973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:57.945336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d74556-db3e-42f0-b774-64d3f1121973 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.945619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d74556-db3e-42f0-b774-64d3f1121973 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:57.945893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d74556-db3e-42f0-b774-64d3f1121973 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:57.946659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d74556-db3e-42f0-b774-64d3f1121973 None admin] RBAC: Adding query filter params (role.id=037adbca67514eda861c68218a60c50a, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=cb6d3abe410746828fb07621ef1a1a5c) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:57.946974 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d74556-db3e-42f0-b774-64d3f1121973 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:57.948639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d74556-db3e-42f0-b774-64d3f1121973 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:57.954815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a46d2-55ce-4384-aae3-076a71660d2e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:57.954958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 307/614] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:29:57 2022] GET /identity/v3/role_assignments?user.id=cb6d3abe410746828fb07621ef1a1a5c&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=037adbca67514eda861c68218a60c50a => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:57.961853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d8fcb-3627-4777-8ef5-4b00a3779f2a None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.962134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d8fcb-3627-4777-8ef5-4b00a3779f2a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:57.962439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d8fcb-3627-4777-8ef5-4b00a3779f2a None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:57.964080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 308/615] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:57.976531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efd1f8-a380-4a3f-8661-d32159b728a0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.976820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efd1f8-a380-4a3f-8661-d32159b728a0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:57.977162 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efd1f8-a380-4a3f-8661-d32159b728a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:57.998496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eee050-9ac6-41a1-b4e0-e94c9750e6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:57.999479 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eee050-9ac6-41a1-b4e0-e94c9750e6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:57.999938 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eee050-9ac6-41a1-b4e0-e94c9750e6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:58.000384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eee050-9ac6-41a1-b4e0-e94c9750e6b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:58.013095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-30efd1f8-a380-4a3f-8661-d32159b728a0 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:58.016434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-30efd1f8-a380-4a3f-8661-d32159b728a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:58.025819 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eee050-9ac6-41a1-b4e0-e94c9750e6b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:58.029974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eee050-9ac6-41a1-b4e0-e94c9750e6b0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-04eee050-9ac6-41a1-b4e0-e94c9750e6b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:58.064914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:58.065947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:58.066897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:58.066897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:58.066897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:58.066897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 308/616] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:58.074021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eee050-9ac6-41a1-b4e0-e94c9750e6b0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:58.112721 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 309/617] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:29:58.157667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:58.189601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efd1f8-a380-4a3f-8661-d32159b728a0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:58.200072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eee050-9ac6-41a1-b4e0-e94c9750e6b0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:58.236047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f505b32-d65f-4c39-9b5b-250466d8248b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:58.236661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f505b32-d65f-4c39-9b5b-250466d8248b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:58.236942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f505b32-d65f-4c39-9b5b-250466d8248b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:58.237227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f505b32-d65f-4c39-9b5b-250466d8248b None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:58.237858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f505b32-d65f-4c39-9b5b-250466d8248b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:58.238182 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f505b32-d65f-4c39-9b5b-250466d8248b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:58.240137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f505b32-d65f-4c39-9b5b-250466d8248b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:58.251330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:58.252649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 309/618] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:58.259281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f505b32-d65f-4c39-9b5b-250466d8248b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:58.318431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efd1f8-a380-4a3f-8661-d32159b728a0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:58.356959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52de67-f5cc-4177-a2e4-c7ea21df8337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:58.358081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52de67-f5cc-4177-a2e4-c7ea21df8337 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:58.358621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52de67-f5cc-4177-a2e4-c7ea21df8337 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:58.359104 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52de67-f5cc-4177-a2e4-c7ea21df8337 None admin] PATH_INFO: `/v3/services/network` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:58.359888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52de67-f5cc-4177-a2e4-c7ea21df8337 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:58.361647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52de67-f5cc-4177-a2e4-c7ea21df8337 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:58.367490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f505b32-d65f-4c39-9b5b-250466d8248b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-fe52de67-f5cc-4177-a2e4-c7ea21df8337 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:58.373958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:58.375001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:29:58.375853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:58.378786 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 310/619] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:58 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:58.384848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52de67-f5cc-4177-a2e4-c7ea21df8337 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:58.392020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:58.403237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e207f4-11e9-4181-bc3c-53cc92a6cc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:58.403868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e207f4-11e9-4181-bc3c-53cc92a6cc91 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:58.404150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e207f4-11e9-4181-bc3c-53cc92a6cc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:58.404419 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e207f4-11e9-4181-bc3c-53cc92a6cc91 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:58.413486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e207f4-11e9-4181-bc3c-53cc92a6cc91 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:58.415021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e207f4-11e9-4181-bc3c-53cc92a6cc91 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-b0e207f4-11e9-4181-bc3c-53cc92a6cc91 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:29:58.428427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:58.429409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:58.430386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:58.430386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:58.430386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:29:58.430386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:29:58.430386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:29:58.430386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:29:58.430386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:29:58.430386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:29:58.430386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:29:58.430386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:58.430386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 310/620] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:29:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:58.437441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e207f4-11e9-4181-bc3c-53cc92a6cc91 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:58.525064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52de67-f5cc-4177-a2e4-c7ea21df8337 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:58.538467 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:58.562785 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e207f4-11e9-4181-bc3c-53cc92a6cc91 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:58.572853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1edfe81-44e6-4668-8f76-85b4cc123424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:58.573469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edfe81-44e6-4668-8f76-85b4cc123424 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:58.573750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edfe81-44e6-4668-8f76-85b4cc123424 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:58.574022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edfe81-44e6-4668-8f76-85b4cc123424 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:58.574699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1edfe81-44e6-4668-8f76-85b4cc123424 None admin] RBAC: Adding query filter params (name=network) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:58.575009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1edfe81-44e6-4668-8f76-85b4cc123424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:58.576366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1edfe81-44e6-4668-8f76-85b4cc123424 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:58.586281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 311/621] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:29:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:29:58.586281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1edfe81-44e6-4668-8f76-85b4cc123424 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:58.615123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598f2ce-84be-4ce0-89d1-83e807fd320d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:58.616162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f2ce-84be-4ce0-89d1-83e807fd320d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:58.616532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f2ce-84be-4ce0-89d1-83e807fd320d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:58.616913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f2ce-84be-4ce0-89d1-83e807fd320d None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:58.617974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598f2ce-84be-4ce0-89d1-83e807fd320d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:58.618526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598f2ce-84be-4ce0-89d1-83e807fd320d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:58.620139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598f2ce-84be-4ce0-89d1-83e807fd320d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:58.642827 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 311/622] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Oct 24 13:29:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:58.644352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598f2ce-84be-4ce0-89d1-83e807fd320d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:58.696431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1edfe81-44e6-4668-8f76-85b4cc123424 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:58.697040 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:58.734585 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418afd2c-b83f-4428-b61b-a708d58b31bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:58.735614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418afd2c-b83f-4428-b61b-a708d58b31bb None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:58.736125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418afd2c-b83f-4428-b61b-a708d58b31bb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:58.736653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418afd2c-b83f-4428-b61b-a708d58b31bb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:58.737440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418afd2c-b83f-4428-b61b-a708d58b31bb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:58.739689 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418afd2c-b83f-4428-b61b-a708d58b31bb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-418afd2c-b83f-4428-b61b-a708d58b31bb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:58.749773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:58.750843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:29:58.751692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:58.760264 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 312/623] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Oct 24 13:29:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:58.761369 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418afd2c-b83f-4428-b61b-a708d58b31bb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:58.762719 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598f2ce-84be-4ce0-89d1-83e807fd320d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:58.800624 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612965b-ad19-4107-9feb-449e5cfe2aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:58.802106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612965b-ad19-4107-9feb-449e5cfe2aaa None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:58.802620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612965b-ad19-4107-9feb-449e5cfe2aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:58.803143 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612965b-ad19-4107-9feb-449e5cfe2aaa None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:58.803995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612965b-ad19-4107-9feb-449e5cfe2aaa None admin] RBAC: Adding query filter params (type=network) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:58.804516 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612965b-ad19-4107-9feb-449e5cfe2aaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:58.806075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612965b-ad19-4107-9feb-449e5cfe2aaa None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:58.810433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:58.812083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 312/624] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:29:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:58.813085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612965b-ad19-4107-9feb-449e5cfe2aaa None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:58.903246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418afd2c-b83f-4428-b61b-a708d58b31bb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:58.939348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc944f-fe29-4ea5-a8a3-1344519d75f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:58.940134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612965b-ad19-4107-9feb-449e5cfe2aaa None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:58.940630 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc944f-fe29-4ea5-a8a3-1344519d75f0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:58.941184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc944f-fe29-4ea5-a8a3-1344519d75f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:58.941733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc944f-fe29-4ea5-a8a3-1344519d75f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:58.949635 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:58.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6531b-f5ac-42e4-ba6d-10144dd9bc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:58.981637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6531b-f5ac-42e4-ba6d-10144dd9bc61 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:58.982111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6531b-f5ac-42e4-ba6d-10144dd9bc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:58.982513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6531b-f5ac-42e4-ba6d-10144dd9bc61 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:58.983230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6531b-f5ac-42e4-ba6d-10144dd9bc61 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:58.983674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6531b-f5ac-42e4-ba6d-10144dd9bc61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:58.985312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6531b-f5ac-42e4-ba6d-10144dd9bc61 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:58.994091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 313/625] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Oct 24 13:29:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:58.995070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6531b-f5ac-42e4-ba6d-10144dd9bc61 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:59.038452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc944f-fe29-4ea5-a8a3-1344519d75f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:59.040075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc944f-fe29-4ea5-a8a3-1344519d75f0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:59.075329 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:59.129712 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6531b-f5ac-42e4-ba6d-10144dd9bc61 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:59.151857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 313/626] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:29:58 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:29:59.153007 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc944f-fe29-4ea5-a8a3-1344519d75f0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:59.178408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb66c4e-d93a-4fed-8d0c-0e2e7f648ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:59.179748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb66c4e-d93a-4fed-8d0c-0e2e7f648ea4 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:59.180182 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb66c4e-d93a-4fed-8d0c-0e2e7f648ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:59.180882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb66c4e-d93a-4fed-8d0c-0e2e7f648ea4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:59.181725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb66c4e-d93a-4fed-8d0c-0e2e7f648ea4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:59.183932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb66c4e-d93a-4fed-8d0c-0e2e7f648ea4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:59.227629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 314/627] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:29:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:59.227629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb66c4e-d93a-4fed-8d0c-0e2e7f648ea4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:59.262613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:59.310971 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc944f-fe29-4ea5-a8a3-1344519d75f0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:59.356781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:59.366783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ba6f1-fe1b-4772-9641-15b73edef1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:59.367153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb66c4e-d93a-4fed-8d0c-0e2e7f648ea4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:59.367662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ba6f1-fe1b-4772-9641-15b73edef1ff None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:59.368040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ba6f1-fe1b-4772-9641-15b73edef1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:59.368409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ba6f1-fe1b-4772-9641-15b73edef1ff None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:59.369185 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ba6f1-fe1b-4772-9641-15b73edef1ff None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:59.369832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ba6f1-fe1b-4772-9641-15b73edef1ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:59.372822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ba6f1-fe1b-4772-9641-15b73edef1ff None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:59.382365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 314/628] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Oct 24 13:29:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:59.384436 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fac76-59bd-4943-802e-dcb380675e43 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:59.386520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fac76-59bd-4943-802e-dcb380675e43 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:59.387638 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fac76-59bd-4943-802e-dcb380675e43 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:59.389207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 315/629] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:29:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:29:59.390216 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fac76-59bd-4943-802e-dcb380675e43 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:59.425299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd35771-2db8-4f16-8903-296508be11df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:59.425940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35771-2db8-4f16-8903-296508be11df None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:59.426238 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35771-2db8-4f16-8903-296508be11df None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:59.426632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35771-2db8-4f16-8903-296508be11df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:59.440441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd35771-2db8-4f16-8903-296508be11df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:59.441875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd35771-2db8-4f16-8903-296508be11df None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-afd35771-2db8-4f16-8903-296508be11df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:29:59.460148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:59.461260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:29:59.462602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:29:59.462602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:29:59.462602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:29:59.462602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 315/630] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:59.462881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ed1f6-abd7-493e-87b4-3f44591fd56d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:59.462958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ed1f6-abd7-493e-87b4-3f44591fd56d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:59.463207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ed1f6-abd7-493e-87b4-3f44591fd56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:59.490353 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8b0ed1f6-abd7-493e-87b4-3f44591fd56d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:59.492221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8b0ed1f6-abd7-493e-87b4-3f44591fd56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:59.543259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fac76-59bd-4943-802e-dcb380675e43 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:59.578206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 316/631] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:29:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:29:59.578206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ed1f6-abd7-493e-87b4-3f44591fd56d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:59.586795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafeba3f-0024-4e61-9636-3742c0b4750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:59.587614 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:59.588515 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafeba3f-0024-4e61-9636-3742c0b4750d None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:59.588861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafeba3f-0024-4e61-9636-3742c0b4750d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:59.589307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafeba3f-0024-4e61-9636-3742c0b4750d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:59.590008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafeba3f-0024-4e61-9636-3742c0b4750d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:59.591395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafeba3f-0024-4e61-9636-3742c0b4750d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:59.616977 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 316/632] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:29:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:59.684231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ed1f6-abd7-493e-87b4-3f44591fd56d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:59.728526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a357e9d-e82d-4cc6-8fcd-e602fa2c17bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:59.729838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a357e9d-e82d-4cc6-8fcd-e602fa2c17bf None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:59.730331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a357e9d-e82d-4cc6-8fcd-e602fa2c17bf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:59.730749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a357e9d-e82d-4cc6-8fcd-e602fa2c17bf None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:59.731608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a357e9d-e82d-4cc6-8fcd-e602fa2c17bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:59.732056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a357e9d-e82d-4cc6-8fcd-e602fa2c17bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:59.733598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a357e9d-e82d-4cc6-8fcd-e602fa2c17bf None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:59.747238 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 317/633] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:59.754320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafeba3f-0024-4e61-9636-3742c0b4750d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:59.810606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:59.871704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafeba3f-0024-4e61-9636-3742c0b4750d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:59.875514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a357e9d-e82d-4cc6-8fcd-e602fa2c17bf None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:59.915751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd580fe1-c221-4ebf-916d-491082137b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:59.916395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd580fe1-c221-4ebf-916d-491082137b74 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:59.916668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd580fe1-c221-4ebf-916d-491082137b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:59.916935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd580fe1-c221-4ebf-916d-491082137b74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:59.924210 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd580fe1-c221-4ebf-916d-491082137b74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:59.925568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd580fe1-c221-4ebf-916d-491082137b74 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-cd580fe1-c221-4ebf-916d-491082137b74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:29:59.937388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:29:59.938514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:29:59.939618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:29:59.939618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:29:59.939618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:29:59.939618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:29:59.939618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:29:59.939618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:29:59.939618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:29:59.939618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:29:59.939618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 317/634] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:29:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:29:59.955390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd580fe1-c221-4ebf-916d-491082137b74 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:59.956263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:00.001397 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a357e9d-e82d-4cc6-8fcd-e602fa2c17bf None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:00.039421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc21034-1009-4ea5-9013-c7da25e08c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:00.040173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc21034-1009-4ea5-9013-c7da25e08c78 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:00.040831 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc21034-1009-4ea5-9013-c7da25e08c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:00.041152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc21034-1009-4ea5-9013-c7da25e08c78 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:00.041902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc21034-1009-4ea5-9013-c7da25e08c78 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:00.043391 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc21034-1009-4ea5-9013-c7da25e08c78 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:00.068667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 318/635] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:29:59 2022] POST /identity/v3/services => generated 269 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:30:00.104500 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd580fe1-c221-4ebf-916d-491082137b74 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:00.118407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:00.164424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4664d-1607-4ad8-b2f0-86a1fe190a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:00.164962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4664d-1607-4ad8-b2f0-86a1fe190a78 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:00.165235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4664d-1607-4ad8-b2f0-86a1fe190a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:00.165748 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4664d-1607-4ad8-b2f0-86a1fe190a78 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:00.166491 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4664d-1607-4ad8-b2f0-86a1fe190a78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:00.166791 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4664d-1607-4ad8-b2f0-86a1fe190a78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:00.168314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4664d-1607-4ad8-b2f0-86a1fe190a78 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:00.184026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 318/636] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:29:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:00.190021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc21034-1009-4ea5-9013-c7da25e08c78 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:00.311826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc21034-1009-4ea5-9013-c7da25e08c78 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:00.338536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:00.353399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc7876-f2ea-40ec-ba24-3ba4e5a13989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:00.354327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc7876-f2ea-40ec-ba24-3ba4e5a13989 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:00.354749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc7876-f2ea-40ec-ba24-3ba4e5a13989 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:00.355158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc7876-f2ea-40ec-ba24-3ba4e5a13989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:00.356102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc7876-f2ea-40ec-ba24-3ba4e5a13989 None admin] RBAC: Adding query filter params (role.id=8b32d66a3cb3483c8ede361ad8637f66, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=0ea0d300a51c41fba4d4193dd2251a50) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:00.356556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc7876-f2ea-40ec-ba24-3ba4e5a13989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:00.358217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc7876-f2ea-40ec-ba24-3ba4e5a13989 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:00.366000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 319/637] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:30:00 2022] GET /identity/v3/role_assignments?user.id=0ea0d300a51c41fba4d4193dd2251a50&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=8b32d66a3cb3483c8ede361ad8637f66 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:00.490758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09db0e3-13a8-4805-bedb-1b7a2c643c67 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:00.491127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09db0e3-13a8-4805-bedb-1b7a2c643c67 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:00.491505 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09db0e3-13a8-4805-bedb-1b7a2c643c67 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:00.493284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 319/638] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:00.499243 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecbd89-6821-4333-bd16-09109b62d00b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:00.499673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecbd89-6821-4333-bd16-09109b62d00b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:00.500046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecbd89-6821-4333-bd16-09109b62d00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:00.519895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-39ecbd89-6821-4333-bd16-09109b62d00b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:00.521170 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-39ecbd89-6821-4333-bd16-09109b62d00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:00.596540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 320/639] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:00 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:00.887332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09db0e3-13a8-4805-bedb-1b7a2c643c67 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:00.994645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09db0e3-13a8-4805-bedb-1b7a2c643c67 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:01.030213 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe8287-e04c-4c9c-9ac7-dc61eff33043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:01.030861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe8287-e04c-4c9c-9ac7-dc61eff33043 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:01.031144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe8287-e04c-4c9c-9ac7-dc61eff33043 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:01.031457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe8287-e04c-4c9c-9ac7-dc61eff33043 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:01.042900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe8287-e04c-4c9c-9ac7-dc61eff33043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-55fe8287-e04c-4c9c-9ac7-dc61eff33043 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:30:01.047693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:01.048604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:01.049829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 320/640] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Oct 24 13:30:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:01.057922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecbd89-6821-4333-bd16-09109b62d00b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:01.169088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecbd89-6821-4333-bd16-09109b62d00b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:01.190533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:01.211700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52c738-238c-4cda-a244-7e61f0aa067e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:01.212342 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52c738-238c-4cda-a244-7e61f0aa067e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:01.212625 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52c738-238c-4cda-a244-7e61f0aa067e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:01.212902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52c738-238c-4cda-a244-7e61f0aa067e None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:01.213575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52c738-238c-4cda-a244-7e61f0aa067e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:01.213878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52c738-238c-4cda-a244-7e61f0aa067e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:01.215293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52c738-238c-4cda-a244-7e61f0aa067e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:01.224629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 321/641] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Oct 24 13:30:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:01.230139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe8287-e04c-4c9c-9ac7-dc61eff33043 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:01.259772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:01.340846 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe8287-e04c-4c9c-9ac7-dc61eff33043 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:01.381796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266c959-e05f-4e04-9c73-602d8e4b05e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:01.382457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266c959-e05f-4e04-9c73-602d8e4b05e3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:01.382732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266c959-e05f-4e04-9c73-602d8e4b05e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:01.382994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266c959-e05f-4e04-9c73-602d8e4b05e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:01.393533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266c959-e05f-4e04-9c73-602d8e4b05e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:01.395039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266c959-e05f-4e04-9c73-602d8e4b05e3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-5266c959-e05f-4e04-9c73-602d8e4b05e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:30:01.410459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:01.412122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:30:01.413020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:30:01.413020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:01.413020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:01.415883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 321/642] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:01.420882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52c738-238c-4cda-a244-7e61f0aa067e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:01.533182 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52c738-238c-4cda-a244-7e61f0aa067e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:01.569269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64aee72-754d-4e7a-b11e-7d59b9fdb012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:01.569877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64aee72-754d-4e7a-b11e-7d59b9fdb012 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:01.570152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64aee72-754d-4e7a-b11e-7d59b9fdb012 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:01.570446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64aee72-754d-4e7a-b11e-7d59b9fdb012 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:01.571078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64aee72-754d-4e7a-b11e-7d59b9fdb012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:01.571436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64aee72-754d-4e7a-b11e-7d59b9fdb012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:01.572785 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64aee72-754d-4e7a-b11e-7d59b9fdb012 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:01.586281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 322/643] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:01.595302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266c959-e05f-4e04-9c73-602d8e4b05e3 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:01.681708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:01.705873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266c959-e05f-4e04-9c73-602d8e4b05e3 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:01.743353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03b549-61b0-405e-aaac-5b48e2a0a7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:01.744006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b549-61b0-405e-aaac-5b48e2a0a7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:01.744287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b549-61b0-405e-aaac-5b48e2a0a7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:01.744562 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03b549-61b0-405e-aaac-5b48e2a0a7d6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:01.752873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03b549-61b0-405e-aaac-5b48e2a0a7d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:01.754336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03b549-61b0-405e-aaac-5b48e2a0a7d6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-da03b549-61b0-405e-aaac-5b48e2a0a7d6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:30:01.766339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:01.767327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:01.768275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:01.768275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:01.768275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:30:01.768275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:30:01.768275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:30:01.768275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:30:01.768275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:30:01.768275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:30:01.768275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:30:01.768275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:01.768275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 322/644] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Oct 24 13:30:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:01.775710 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64aee72-754d-4e7a-b11e-7d59b9fdb012 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:01.820430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:01.883955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64aee72-754d-4e7a-b11e-7d59b9fdb012 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:01.920547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da518651-7c17-428c-b47c-064b2034943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:01.921510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da518651-7c17-428c-b47c-064b2034943c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:01.921996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da518651-7c17-428c-b47c-064b2034943c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:01.922524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da518651-7c17-428c-b47c-064b2034943c None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:01.923466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da518651-7c17-428c-b47c-064b2034943c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:01.923972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da518651-7c17-428c-b47c-064b2034943c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:01.925598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da518651-7c17-428c-b47c-064b2034943c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:01.944910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 323/645] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Oct 24 13:30:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:01.951166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03b549-61b0-405e-aaac-5b48e2a0a7d6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:02.065612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03b549-61b0-405e-aaac-5b48e2a0a7d6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:02.093404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73f355-6bbf-4b8f-b2c5-6f8780b58e4f None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.094530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73f355-6bbf-4b8f-b2c5-6f8780b58e4f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.095013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73f355-6bbf-4b8f-b2c5-6f8780b58e4f None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.097362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 324/646] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:02.103336 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075f7a-e191-4a2f-b86c-7269ac6b2a63 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.103915 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075f7a-e191-4a2f-b86c-7269ac6b2a63 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.104467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075f7a-e191-4a2f-b86c-7269ac6b2a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.104782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e25177-f592-4e68-ac32-c1f104733035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:02.105421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25177-f592-4e68-ac32-c1f104733035 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.105701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25177-f592-4e68-ac32-c1f104733035 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.105975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e25177-f592-4e68-ac32-c1f104733035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.112997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:02.119851 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e25177-f592-4e68-ac32-c1f104733035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:02.121301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e25177-f592-4e68-ac32-c1f104733035 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:02.128862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6b075f7a-e191-4a2f-b86c-7269ac6b2a63 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:02.131253 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6b075f7a-e191-4a2f-b86c-7269ac6b2a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-39e25177-f592-4e68-ac32-c1f104733035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:30:02.138188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:02.139124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:30:02.140014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:30:02.140014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:02.140014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:02.140014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 323/647] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:02.147735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e25177-f592-4e68-ac32-c1f104733035 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:02.217136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 325/648] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:02 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:02.268400 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e25177-f592-4e68-ac32-c1f104733035 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:02.306463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963fe5c-9b47-4332-a11b-15830f29c31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:02.307472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963fe5c-9b47-4332-a11b-15830f29c31b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.307960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963fe5c-9b47-4332-a11b-15830f29c31b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.308437 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963fe5c-9b47-4332-a11b-15830f29c31b None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.309280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963fe5c-9b47-4332-a11b-15830f29c31b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:02.309785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963fe5c-9b47-4332-a11b-15830f29c31b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:02.311381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963fe5c-9b47-4332-a11b-15830f29c31b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:02.324771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 324/649] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:02.334559 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075f7a-e191-4a2f-b86c-7269ac6b2a63 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:02.351259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88aca8d-a8ec-4723-ab84-ab69cec0fb81 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.351532 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88aca8d-a8ec-4723-ab84-ab69cec0fb81 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.351807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88aca8d-a8ec-4723-ab84-ab69cec0fb81 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.353390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 325/650] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:30:02.358227 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512978a9-29bd-4377-936b-ce9c39f0d7b9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.358513 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512978a9-29bd-4377-936b-ce9c39f0d7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.358782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512978a9-29bd-4377-936b-ce9c39f0d7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.362270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:02.378297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-512978a9-29bd-4377-936b-ce9c39f0d7b9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:02.379714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-512978a9-29bd-4377-936b-ce9c39f0d7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:02.432362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:02.451907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075f7a-e191-4a2f-b86c-7269ac6b2a63 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:02.461480 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 326/651] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:02 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:02.494468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f534194-fe4e-4e57-b628-52ab398d7d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:02.495495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f534194-fe4e-4e57-b628-52ab398d7d66 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.495997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f534194-fe4e-4e57-b628-52ab398d7d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.496469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f534194-fe4e-4e57-b628-52ab398d7d66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.503764 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f534194-fe4e-4e57-b628-52ab398d7d66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:02.505621 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f534194-fe4e-4e57-b628-52ab398d7d66 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-5f534194-fe4e-4e57-b628-52ab398d7d66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:30:02.517452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:02.518370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:02.519294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:30:02.519294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:30:02.519294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:30:02.519294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:30:02.519294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:30:02.519294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:30:02.519294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:30:02.519294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:02.521786 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512978a9-29bd-4377-936b-ce9c39f0d7b9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:02.523131 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 326/652] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:30:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:02.529156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f534194-fe4e-4e57-b628-52ab398d7d66 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:02.630399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:02.637130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512978a9-29bd-4377-936b-ce9c39f0d7b9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:02.644003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f534194-fe4e-4e57-b628-52ab398d7d66 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:02.675309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0755e-7bd4-4c98-bddf-67b8b51c17d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:02.675984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0755e-7bd4-4c98-bddf-67b8b51c17d8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.676267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0755e-7bd4-4c98-bddf-67b8b51c17d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.677272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0755e-7bd4-4c98-bddf-67b8b51c17d8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.677888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0755e-7bd4-4c98-bddf-67b8b51c17d8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:02.679282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0755e-7bd4-4c98-bddf-67b8b51c17d8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:02.682644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77444bf-0260-4ccd-b540-2ccd3ba50841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:02.683325 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77444bf-0260-4ccd-b540-2ccd3ba50841 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.683530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77444bf-0260-4ccd-b540-2ccd3ba50841 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.683800 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77444bf-0260-4ccd-b540-2ccd3ba50841 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.684485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77444bf-0260-4ccd-b540-2ccd3ba50841 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:02.684786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77444bf-0260-4ccd-b540-2ccd3ba50841 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:02.686208 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77444bf-0260-4ccd-b540-2ccd3ba50841 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-88e0755e-7bd4-4c98-bddf-67b8b51c17d8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:02.693546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:02.694772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:02.701259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 327/653] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:30:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:02.702061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcd938-f7e9-4207-aef1-5a0e4487e053 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.702061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcd938-f7e9-4207-aef1-5a0e4487e053 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.702061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcd938-f7e9-4207-aef1-5a0e4487e053 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.702061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 328/654] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:02.702061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dcd938-f7e9-4207-aef1-5a0e4487e053 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:02.703469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 327/655] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:30:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:02.709417 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6caa39-457b-4f55-ac3b-e116cbd43cc9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.709710 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6caa39-457b-4f55-ac3b-e116cbd43cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.709985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6caa39-457b-4f55-ac3b-e116cbd43cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.727951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-eb6caa39-457b-4f55-ac3b-e116cbd43cc9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:02.729123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-eb6caa39-457b-4f55-ac3b-e116cbd43cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:02.752244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:02.796916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 328/656] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:02 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:02.797853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6caa39-457b-4f55-ac3b-e116cbd43cc9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:02.821415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dcd938-f7e9-4207-aef1-5a0e4487e053 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:02.857596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9605e15-b180-44b3-9d8d-3ffa34de22f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:02.858231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9605e15-b180-44b3-9d8d-3ffa34de22f2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.858509 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9605e15-b180-44b3-9d8d-3ffa34de22f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.858773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9605e15-b180-44b3-9d8d-3ffa34de22f2 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.859390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9605e15-b180-44b3-9d8d-3ffa34de22f2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:02.859743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9605e15-b180-44b3-9d8d-3ffa34de22f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:02.861092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9605e15-b180-44b3-9d8d-3ffa34de22f2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:02.867106 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 329/657] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:30:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:02.867944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9605e15-b180-44b3-9d8d-3ffa34de22f2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:02.887120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:02.910680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6caa39-457b-4f55-ac3b-e116cbd43cc9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:02.947690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3b817-ee59-4a61-88e3-1ef7e0ca5a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:02.948352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b817-ee59-4a61-88e3-1ef7e0ca5a92 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:02.948631 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b817-ee59-4a61-88e3-1ef7e0ca5a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:02.948907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b817-ee59-4a61-88e3-1ef7e0ca5a92 None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/cb6d3abe410746828fb07621ef1a1a5c/roles/79ebc57a5a9041d08f54488fb7845f5e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:02.951117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3b817-ee59-4a61-88e3-1ef7e0ca5a92 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=cb6d3abe410746828fb07621ef1a1a5c, role_id=79ebc57a5a9041d08f54488fb7845f5e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:02.952538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3b817-ee59-4a61-88e3-1ef7e0ca5a92 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:02.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 329/658] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:30:02 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/cb6d3abe410746828fb07621ef1a1a5c/roles/79ebc57a5a9041d08f54488fb7845f5e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:30:02.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3b817-ee59-4a61-88e3-1ef7e0ca5a92 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:02.982660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9605e15-b180-44b3-9d8d-3ffa34de22f2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:02.994433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:03.032338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e35f80-5195-443d-8131-ba484ea72aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:03.033479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e35f80-5195-443d-8131-ba484ea72aad None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:03.033821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e35f80-5195-443d-8131-ba484ea72aad None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:03.034158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e35f80-5195-443d-8131-ba484ea72aad None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:03.035250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e35f80-5195-443d-8131-ba484ea72aad None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:03.037879 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e35f80-5195-443d-8131-ba484ea72aad None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-87e35f80-5195-443d-8131-ba484ea72aad None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:03.051380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:03.052561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:03.053455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 330/659] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Oct 24 13:30:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:03.059061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e35f80-5195-443d-8131-ba484ea72aad None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:03.105282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3b817-ee59-4a61-88e3-1ef7e0ca5a92 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:03.144661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e80fb-902e-43c5-a485-f1e65254f9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:03.145881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e80fb-902e-43c5-a485-f1e65254f9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:03.146155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e80fb-902e-43c5-a485-f1e65254f9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:03.146456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e80fb-902e-43c5-a485-f1e65254f9d8 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:03.147112 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e80fb-902e-43c5-a485-f1e65254f9d8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:03.147415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e80fb-902e-43c5-a485-f1e65254f9d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:03.148804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e80fb-902e-43c5-a485-f1e65254f9d8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:03.154202 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 330/660] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:30:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:03.155016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e80fb-902e-43c5-a485-f1e65254f9d8 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:03.157900 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:03.185501 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e35f80-5195-443d-8131-ba484ea72aad None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:03.239873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e535c6a-fe87-4f3b-924a-dc83849dc53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:03.240510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535c6a-fe87-4f3b-924a-dc83849dc53c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:03.240797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535c6a-fe87-4f3b-924a-dc83849dc53c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:03.241113 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535c6a-fe87-4f3b-924a-dc83849dc53c None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:03.241766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e535c6a-fe87-4f3b-924a-dc83849dc53c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:03.242069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e535c6a-fe87-4f3b-924a-dc83849dc53c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:03.243592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e535c6a-fe87-4f3b-924a-dc83849dc53c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:03.249210 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 331/661] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Oct 24 13:30:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:03.255366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e535c6a-fe87-4f3b-924a-dc83849dc53c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:03.294198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:03.309927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e80fb-902e-43c5-a485-f1e65254f9d8 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:03.353582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14d02e-dea6-4817-82a2-f58e82ee52dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:03.354648 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14d02e-dea6-4817-82a2-f58e82ee52dc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:03.355156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14d02e-dea6-4817-82a2-f58e82ee52dc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:03.355639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14d02e-dea6-4817-82a2-f58e82ee52dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:03.356305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14d02e-dea6-4817-82a2-f58e82ee52dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:03.357665 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14d02e-dea6-4817-82a2-f58e82ee52dc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-0a14d02e-dea6-4817-82a2-f58e82ee52dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:03.370285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:03.371321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:03.372169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 331/662] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Oct 24 13:30:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:03.375009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e535c6a-fe87-4f3b-924a-dc83849dc53c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:03.380773 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14d02e-dea6-4817-82a2-f58e82ee52dc None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:03.418419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620347d-2065-4391-9f83-afdad614b566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:03.419096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620347d-2065-4391-9f83-afdad614b566 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:03.419375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620347d-2065-4391-9f83-afdad614b566 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:03.419640 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620347d-2065-4391-9f83-afdad614b566 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:03.420264 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620347d-2065-4391-9f83-afdad614b566 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:03.420797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620347d-2065-4391-9f83-afdad614b566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:03.422888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620347d-2065-4391-9f83-afdad614b566 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:03.437096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 332/663] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Oct 24 13:30:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:03.439078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620347d-2065-4391-9f83-afdad614b566 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:03.464519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:03.517139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14d02e-dea6-4817-82a2-f58e82ee52dc None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:03.558114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620347d-2065-4391-9f83-afdad614b566 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:03.560715 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3844c1-f94c-4a4f-8483-ae599fae2566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:03.561329 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3844c1-f94c-4a4f-8483-ae599fae2566 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:03.561606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3844c1-f94c-4a4f-8483-ae599fae2566 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:03.561872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3844c1-f94c-4a4f-8483-ae599fae2566 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:03.562581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3844c1-f94c-4a4f-8483-ae599fae2566 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:03.562882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3844c1-f94c-4a4f-8483-ae599fae2566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:03.564377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:03.564574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3844c1-f94c-4a4f-8483-ae599fae2566 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:03.572458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 332/664] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:30:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:03.577622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3844c1-f94c-4a4f-8483-ae599fae2566 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:03.609594 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1d9e5-b41e-40d5-bc58-077005cc82a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:03.610280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1d9e5-b41e-40d5-bc58-077005cc82a8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:03.610562 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1d9e5-b41e-40d5-bc58-077005cc82a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:03.610886 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1d9e5-b41e-40d5-bc58-077005cc82a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:03.611581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1d9e5-b41e-40d5-bc58-077005cc82a8 None admin] RBAC: Adding query filter params (interface=public, service_id=cd472cdefc0444bf91b411070b4ee28b, region_id=RegionOne) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:03.611890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1d9e5-b41e-40d5-bc58-077005cc82a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:03.613369 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1d9e5-b41e-40d5-bc58-077005cc82a8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:03.623438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 333/665] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Oct 24 13:30:03 2022] GET /identity/v3/endpoints?service_id=cd472cdefc0444bf91b411070b4ee28b&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:03.624834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1d9e5-b41e-40d5-bc58-077005cc82a8 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:03.698060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3844c1-f94c-4a4f-8483-ae599fae2566 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:03.731142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:03.739842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f67b6-a524-4837-8666-03695dc83cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:03.740473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f67b6-a524-4837-8666-03695dc83cff None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:03.740717 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f67b6-a524-4837-8666-03695dc83cff None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:03.740984 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f67b6-a524-4837-8666-03695dc83cff None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:03.741607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f67b6-a524-4837-8666-03695dc83cff None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:03.741899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f67b6-a524-4837-8666-03695dc83cff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:03.743959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f67b6-a524-4837-8666-03695dc83cff None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:03.746680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1d9e5-b41e-40d5-bc58-077005cc82a8 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:03.755885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 333/666] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:30:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:03.792456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a58fa-e9d1-4ec0-a471-f5e96c6f6b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:03.793077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a58fa-e9d1-4ec0-a471-f5e96c6f6b19 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:03.793356 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a58fa-e9d1-4ec0-a471-f5e96c6f6b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:03.793623 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a58fa-e9d1-4ec0-a471-f5e96c6f6b19 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:03.794275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a58fa-e9d1-4ec0-a471-f5e96c6f6b19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:03.795662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a58fa-e9d1-4ec0-a471-f5e96c6f6b19 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:03.810429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 334/667] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Oct 24 13:30:03 2022] GET /identity/v3/services => generated 1029 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:30:05.217148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bb9d4-0608-4bed-a3b3-889074417a81 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:05.217575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bb9d4-0608-4bed-a3b3-889074417a81 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:05.217757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bb9d4-0608-4bed-a3b3-889074417a81 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:05.219292 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 334/668] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:05.225270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf17a67-28cb-416d-89a9-93b860a8eddf None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:05.225703 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf17a67-28cb-416d-89a9-93b860a8eddf None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:05.225891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf17a67-28cb-416d-89a9-93b860a8eddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:05.244617 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-8bf17a67-28cb-416d-89a9-93b860a8eddf None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:05.245775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8bf17a67-28cb-416d-89a9-93b860a8eddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:05.269280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:05.320104 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 335/669] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:05 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:05.447531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac947cf-bcad-427c-a843-39b075e209ef None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:05.447847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac947cf-bcad-427c-a843-39b075e209ef None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:05.448073 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac947cf-bcad-427c-a843-39b075e209ef None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:05.449652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 335/670] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:05.454898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734038c5-66f2-476a-99aa-c76e9b308d00 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:05.455196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734038c5-66f2-476a-99aa-c76e9b308d00 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:05.455470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734038c5-66f2-476a-99aa-c76e9b308d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:05.475462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-734038c5-66f2-476a-99aa-c76e9b308d00 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:05.476537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-734038c5-66f2-476a-99aa-c76e9b308d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:05.550733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 336/671] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:05 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:05.599727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac947cf-bcad-427c-a843-39b075e209ef None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:05.709372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac947cf-bcad-427c-a843-39b075e209ef None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:05.743972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d679e4d-922c-465e-88b0-42f07f00f8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:05.744518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d679e4d-922c-465e-88b0-42f07f00f8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:05.744796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d679e4d-922c-465e-88b0-42f07f00f8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:05.745061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d679e4d-922c-465e-88b0-42f07f00f8c1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:05.756589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d679e4d-922c-465e-88b0-42f07f00f8c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-7d679e4d-922c-465e-88b0-42f07f00f8c1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:30:05.761529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:05.762584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:05.763455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 336/672] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Oct 24 13:30:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:05.768506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734038c5-66f2-476a-99aa-c76e9b308d00 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:05.859076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d679e4d-922c-465e-88b0-42f07f00f8c1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:05.862251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:05.888890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734038c5-66f2-476a-99aa-c76e9b308d00 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:05.935745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6180-283c-4201-954b-d3cd8f1d7c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:05.936360 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6180-283c-4201-954b-d3cd8f1d7c7c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:05.936773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6180-283c-4201-954b-d3cd8f1d7c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:05.937084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6180-283c-4201-954b-d3cd8f1d7c7c None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:05.937854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b6180-283c-4201-954b-d3cd8f1d7c7c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:05.938186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b6180-283c-4201-954b-d3cd8f1d7c7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:05.939089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:05.939857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b6180-283c-4201-954b-d3cd8f1d7c7c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:05.956878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 337/673] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Oct 24 13:30:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:05.959567 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6180-283c-4201-954b-d3cd8f1d7c7c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:06.002734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d679e4d-922c-465e-88b0-42f07f00f8c1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:06.038797 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191646d-b208-47dd-a3d8-51448de03ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:06.039546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191646d-b208-47dd-a3d8-51448de03ae5 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.039832 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191646d-b208-47dd-a3d8-51448de03ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.040111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191646d-b208-47dd-a3d8-51448de03ae5 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.041029 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191646d-b208-47dd-a3d8-51448de03ae5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:06.042440 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191646d-b208-47dd-a3d8-51448de03ae5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:06.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 337/674] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:30:05 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:06.059786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74818335-2c29-400a-bbbf-ec32edeb5b8d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.060072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74818335-2c29-400a-bbbf-ec32edeb5b8d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.060346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74818335-2c29-400a-bbbf-ec32edeb5b8d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.061808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 338/675] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:06.069252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e8b5-7e4a-456b-964d-bfd25c8f763d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.069649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e8b5-7e4a-456b-964d-bfd25c8f763d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.069982 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e8b5-7e4a-456b-964d-bfd25c8f763d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.079488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6180-283c-4201-954b-d3cd8f1d7c7c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:06.093756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0c94e8b5-7e4a-456b-964d-bfd25c8f763d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:06.095319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0c94e8b5-7e4a-456b-964d-bfd25c8f763d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:06.124397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374b9a7-5ae1-409e-af5e-a2112903b72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:06.125042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374b9a7-5ae1-409e-af5e-a2112903b72f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.125316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374b9a7-5ae1-409e-af5e-a2112903b72f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.125577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374b9a7-5ae1-409e-af5e-a2112903b72f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.138075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374b9a7-5ae1-409e-af5e-a2112903b72f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:06.139567 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374b9a7-5ae1-409e-af5e-a2112903b72f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:06.147158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-e374b9a7-5ae1-409e-af5e-a2112903b72f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:30:06.156717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:06.157763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:30:06.159485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:30:06.159485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:06.159485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:06.159485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 338/676] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:06.159808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c9263-eac6-4c26-af22-5b499abe59bd None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.159808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c9263-eac6-4c26-af22-5b499abe59bd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.160125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c9263-eac6-4c26-af22-5b499abe59bd None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.161827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 339/677] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:06.167427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c9263-eac6-4c26-af22-5b499abe59bd None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:06.198193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 339/678] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:06.198193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1ff2b-66e9-4f6e-a4d7-9aa7e79f812e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.198193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1ff2b-66e9-4f6e-a4d7-9aa7e79f812e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.198193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1ff2b-66e9-4f6e-a4d7-9aa7e79f812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.219462 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-46f1ff2b-66e9-4f6e-a4d7-9aa7e79f812e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:06.220678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-46f1ff2b-66e9-4f6e-a4d7-9aa7e79f812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:06.270467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:06.310053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c9263-eac6-4c26-af22-5b499abe59bd None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:06.317667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 340/679] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:06.352389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d86a3a-157a-459e-a4d0-c7847fc43639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:06.353251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d86a3a-157a-459e-a4d0-c7847fc43639 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.353683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d86a3a-157a-459e-a4d0-c7847fc43639 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.354105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d86a3a-157a-459e-a4d0-c7847fc43639 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.355001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d86a3a-157a-459e-a4d0-c7847fc43639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:06.355451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d86a3a-157a-459e-a4d0-c7847fc43639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:06.356955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d86a3a-157a-459e-a4d0-c7847fc43639 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:06.370307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 340/680] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:06.382250 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1ff2b-66e9-4f6e-a4d7-9aa7e79f812e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:06.481208 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d86a3a-157a-459e-a4d0-c7847fc43639 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:06.482879 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1ff2b-66e9-4f6e-a4d7-9aa7e79f812e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:06.517842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e955496-dc29-4c40-9f3c-6839bf4217de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:06.518548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e955496-dc29-4c40-9f3c-6839bf4217de None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.518822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e955496-dc29-4c40-9f3c-6839bf4217de None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.519088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e955496-dc29-4c40-9f3c-6839bf4217de None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.527550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e955496-dc29-4c40-9f3c-6839bf4217de None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:06.529132 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e955496-dc29-4c40-9f3c-6839bf4217de None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-8e955496-dc29-4c40-9f3c-6839bf4217de None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:30:06.541327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:06.542321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:06.543271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:06.543271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:06.543271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:30:06.543271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:30:06.543271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:30:06.543271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:30:06.543271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:30:06.543271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:30:06.543271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:30:06.543271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:06.543271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 341/681] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Oct 24 13:30:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:06.548280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e955496-dc29-4c40-9f3c-6839bf4217de None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:06.562574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:06.584844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:06.605603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d86a3a-157a-459e-a4d0-c7847fc43639 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:06.643415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d878e34-22c5-4632-ac18-b2bf9acc2a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:06.644019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d878e34-22c5-4632-ac18-b2bf9acc2a6c None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.644305 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d878e34-22c5-4632-ac18-b2bf9acc2a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.644578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d878e34-22c5-4632-ac18-b2bf9acc2a6c None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.645796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d878e34-22c5-4632-ac18-b2bf9acc2a6c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:06.647422 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d878e34-22c5-4632-ac18-b2bf9acc2a6c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:06.655377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e955496-dc29-4c40-9f3c-6839bf4217de None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:06.673415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 341/682] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:30:06 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:30:06.673415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d878e34-22c5-4632-ac18-b2bf9acc2a6c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:06.712974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cceaf0b-1d4b-4d5c-8090-3e4841b10147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:06.713848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceaf0b-1d4b-4d5c-8090-3e4841b10147 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.714336 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceaf0b-1d4b-4d5c-8090-3e4841b10147 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.714749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceaf0b-1d4b-4d5c-8090-3e4841b10147 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.715674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cceaf0b-1d4b-4d5c-8090-3e4841b10147 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:06.716162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cceaf0b-1d4b-4d5c-8090-3e4841b10147 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:06.717832 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cceaf0b-1d4b-4d5c-8090-3e4841b10147 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:06.744951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 342/683] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Oct 24 13:30:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:06.746400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cceaf0b-1d4b-4d5c-8090-3e4841b10147 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:06.791755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d878e34-22c5-4632-ac18-b2bf9acc2a6c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:06.812228 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:06.835314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146efc3-fed3-485d-9376-44bb7d4473ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:06.836074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146efc3-fed3-485d-9376-44bb7d4473ed None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.836420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146efc3-fed3-485d-9376-44bb7d4473ed None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.836759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146efc3-fed3-485d-9376-44bb7d4473ed None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.837426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146efc3-fed3-485d-9376-44bb7d4473ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:06.838998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146efc3-fed3-485d-9376-44bb7d4473ed None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-3146efc3-fed3-485d-9376-44bb7d4473ed None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:06.850091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:06.851304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:30:06.852430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:06.853883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 342/684] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Oct 24 13:30:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:06.858393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:06.861109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146efc3-fed3-485d-9376-44bb7d4473ed None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:06.879020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cceaf0b-1d4b-4d5c-8090-3e4841b10147 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:06.923011 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b46419-eafe-4b4c-83cb-203784e3ffdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:06.923832 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b46419-eafe-4b4c-83cb-203784e3ffdd None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:06.924497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b46419-eafe-4b4c-83cb-203784e3ffdd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:06.924769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b46419-eafe-4b4c-83cb-203784e3ffdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:06.984857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146efc3-fed3-485d-9376-44bb7d4473ed None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:07.010719 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b46419-eafe-4b4c-83cb-203784e3ffdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:07.013798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b46419-eafe-4b4c-83cb-203784e3ffdd None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:07.036953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa855b-39fd-45e4-a64e-a4b2401e9ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:07.037694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa855b-39fd-45e4-a64e-a4b2401e9ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:07.038046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa855b-39fd-45e4-a64e-a4b2401e9ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:07.038967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa855b-39fd-45e4-a64e-a4b2401e9ef2 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:07.039680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa855b-39fd-45e4-a64e-a4b2401e9ef2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:07.040055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa855b-39fd-45e4-a64e-a4b2401e9ef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:07.042417 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa855b-39fd-45e4-a64e-a4b2401e9ef2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:07.054639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 343/685] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Oct 24 13:30:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:07.056178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa855b-39fd-45e4-a64e-a4b2401e9ef2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:07.115608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:07.116907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:07.135992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 343/686] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:30:06 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:30:07.143292 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b46419-eafe-4b4c-83cb-203784e3ffdd None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:07.185717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa855b-39fd-45e4-a64e-a4b2401e9ef2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:07.240763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f8d49-c8d9-4225-996f-e60a1562e58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:07.242328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f8d49-c8d9-4225-996f-e60a1562e58b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:07.242837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f8d49-c8d9-4225-996f-e60a1562e58b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:07.243305 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f8d49-c8d9-4225-996f-e60a1562e58b None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:07.244577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f8d49-c8d9-4225-996f-e60a1562e58b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:07.245482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f8d49-c8d9-4225-996f-e60a1562e58b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:07.247394 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f8d49-c8d9-4225-996f-e60a1562e58b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:07.254965 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 344/687] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Oct 24 13:30:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:07.264465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f8d49-c8d9-4225-996f-e60a1562e58b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:07.270099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b46419-eafe-4b4c-83cb-203784e3ffdd None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:07.313324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3a61b-bef9-440e-9336-22946a4e893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:07.314411 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3a61b-bef9-440e-9336-22946a4e893c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:07.314869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3a61b-bef9-440e-9336-22946a4e893c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:07.315585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3a61b-bef9-440e-9336-22946a4e893c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:07.328099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3a61b-bef9-440e-9336-22946a4e893c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:07.329726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3a61b-bef9-440e-9336-22946a4e893c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-90c3a61b-bef9-440e-9336-22946a4e893c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:30:07.345954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:07.347196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:30:07.348613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:30:07.348613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:07.348613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:07.350714 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 344/688] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:07.359838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3a61b-bef9-440e-9336-22946a4e893c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:07.369726 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f8d49-c8d9-4225-996f-e60a1562e58b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:07.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:07.409494 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bd451-66f4-45c2-b94a-61245b974c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:07.410114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd451-66f4-45c2-b94a-61245b974c67 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:07.410421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd451-66f4-45c2-b94a-61245b974c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:07.411263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd451-66f4-45c2-b94a-61245b974c67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:07.412091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bd451-66f4-45c2-b94a-61245b974c67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:07.413511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bd451-66f4-45c2-b94a-61245b974c67 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:07.417076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:07.440747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 345/689] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:30:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:07.502019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3a61b-bef9-440e-9336-22946a4e893c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:07.551923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34090e4f-366e-43e0-87b3-811f92b6ad2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:07.552872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34090e4f-366e-43e0-87b3-811f92b6ad2c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:07.553296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34090e4f-366e-43e0-87b3-811f92b6ad2c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:07.553755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34090e4f-366e-43e0-87b3-811f92b6ad2c None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:07.554615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34090e4f-366e-43e0-87b3-811f92b6ad2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:07.555121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34090e4f-366e-43e0-87b3-811f92b6ad2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:07.556707 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34090e4f-366e-43e0-87b3-811f92b6ad2c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:07.571420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 345/690] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:07.577367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bd451-66f4-45c2-b94a-61245b974c67 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:07.694501 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bd451-66f4-45c2-b94a-61245b974c67 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:07.729948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c0311-1eef-41f9-84f6-bbd82dac3a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:07.730979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c0311-1eef-41f9-84f6-bbd82dac3a07 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:07.731432 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c0311-1eef-41f9-84f6-bbd82dac3a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:07.731889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c0311-1eef-41f9-84f6-bbd82dac3a07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:07.739274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c0311-1eef-41f9-84f6-bbd82dac3a07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:07.740902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c0311-1eef-41f9-84f6-bbd82dac3a07 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-943c0311-1eef-41f9-84f6-bbd82dac3a07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:30:07.752296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:07.753190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:07.754072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:30:07.754072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:30:07.754072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:30:07.754072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:30:07.754072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:30:07.754072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:30:07.754072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:30:07.754072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:07.756865 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 346/691] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:30:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:07.762665 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34090e4f-366e-43e0-87b3-811f92b6ad2c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:07.864438 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:07.871343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34090e4f-366e-43e0-87b3-811f92b6ad2c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:07.906634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707d365-2975-4435-a4f2-25b8a944d5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:07.907520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707d365-2975-4435-a4f2-25b8a944d5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:07.907943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707d365-2975-4435-a4f2-25b8a944d5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:07.908350 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707d365-2975-4435-a4f2-25b8a944d5e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:07.909168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707d365-2975-4435-a4f2-25b8a944d5e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:07.909608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707d365-2975-4435-a4f2-25b8a944d5e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:07.911717 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707d365-2975-4435-a4f2-25b8a944d5e8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:07.924351 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 346/692] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:30:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:07.931083 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c0311-1eef-41f9-84f6-bbd82dac3a07 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:07.961742 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:08.044247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c0311-1eef-41f9-84f6-bbd82dac3a07 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:08.080435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42004ad-001c-45d9-a11b-459b90b95d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:08.081046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42004ad-001c-45d9-a11b-459b90b95d7b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:08.081328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42004ad-001c-45d9-a11b-459b90b95d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:08.081605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42004ad-001c-45d9-a11b-459b90b95d7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:08.082359 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42004ad-001c-45d9-a11b-459b90b95d7b None admin] RBAC: Adding query filter params (role.id=79ebc57a5a9041d08f54488fb7845f5e, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=cb6d3abe410746828fb07621ef1a1a5c) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:08.083327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42004ad-001c-45d9-a11b-459b90b95d7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:08.085042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42004ad-001c-45d9-a11b-459b90b95d7b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:08.091519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 347/693] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:30:07 2022] GET /identity/v3/role_assignments?user.id=cb6d3abe410746828fb07621ef1a1a5c&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=79ebc57a5a9041d08f54488fb7845f5e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:08.343170 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205672e-96e9-404b-b7be-749019cd0f7b None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:08.343694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205672e-96e9-404b-b7be-749019cd0f7b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:08.343694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205672e-96e9-404b-b7be-749019cd0f7b None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:08.345273 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 347/694] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:08.351605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5adac9-adb1-4526-a98c-7fe67ff56ebd None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:08.351896 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5adac9-adb1-4526-a98c-7fe67ff56ebd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:08.352173 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5adac9-adb1-4526-a98c-7fe67ff56ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:08.371239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5b5adac9-adb1-4526-a98c-7fe67ff56ebd None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:08.372417 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5b5adac9-adb1-4526-a98c-7fe67ff56ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:08.443169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 348/695] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:08.715281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205672e-96e9-404b-b7be-749019cd0f7b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:08.821318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205672e-96e9-404b-b7be-749019cd0f7b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:08.859924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe9a8d-8695-49d6-81ae-346872cf0702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:08.860579 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9a8d-8695-49d6-81ae-346872cf0702 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:08.860923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9a8d-8695-49d6-81ae-346872cf0702 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:08.861200 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9a8d-8695-49d6-81ae-346872cf0702 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:08.861795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe9a8d-8695-49d6-81ae-346872cf0702 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:08.863248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe9a8d-8695-49d6-81ae-346872cf0702 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-1bbe9a8d-8695-49d6-81ae-346872cf0702 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:08.873510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:08.874645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:30:08.875504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:08.880508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 348/696] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:30:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:08.883325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5adac9-adb1-4526-a98c-7fe67ff56ebd None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:08.905440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:08.970964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461161e-60cb-40a9-bbdb-00850de8f7c9 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:08.971149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461161e-60cb-40a9-bbdb-00850de8f7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:08.971423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461161e-60cb-40a9-bbdb-00850de8f7c9 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:08.973109 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 349/697] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:30:08.979527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36facd0f-0856-446a-82fb-d4d19c64b793 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:08.979811 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36facd0f-0856-446a-82fb-d4d19c64b793 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:08.980085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36facd0f-0856-446a-82fb-d4d19c64b793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:08.989287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5adac9-adb1-4526-a98c-7fe67ff56ebd None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:09.002945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-36facd0f-0856-446a-82fb-d4d19c64b793 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:09.005326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-36facd0f-0856-446a-82fb-d4d19c64b793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:09.029434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cbd35-6d73-418d-a08f-74a0864a9380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:09.030149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cbd35-6d73-418d-a08f-74a0864a9380 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:09.030471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cbd35-6d73-418d-a08f-74a0864a9380 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:09.030736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cbd35-6d73-418d-a08f-74a0864a9380 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:09.031787 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cbd35-6d73-418d-a08f-74a0864a9380 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:09.032087 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cbd35-6d73-418d-a08f-74a0864a9380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:09.033711 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cbd35-6d73-418d-a08f-74a0864a9380 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:09.044459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 349/698] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:30:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:09.044631 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:09.046151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cbd35-6d73-418d-a08f-74a0864a9380 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:09.105279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 350/699] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:09.159178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:09.163450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cbd35-6d73-418d-a08f-74a0864a9380 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:09.199967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2229a-b96b-4a68-95d1-45ed22b9bb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:09.200878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2229a-b96b-4a68-95d1-45ed22b9bb1f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:09.201327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2229a-b96b-4a68-95d1-45ed22b9bb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:09.201742 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2229a-b96b-4a68-95d1-45ed22b9bb1f None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:09.202535 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2229a-b96b-4a68-95d1-45ed22b9bb1f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:09.203024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2229a-b96b-4a68-95d1-45ed22b9bb1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:09.204535 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2229a-b96b-4a68-95d1-45ed22b9bb1f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:09.210232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 350/700] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:30:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:09.218994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36facd0f-0856-446a-82fb-d4d19c64b793 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:09.326232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36facd0f-0856-446a-82fb-d4d19c64b793 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:09.365855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587676-a06d-4bda-844e-72da64011b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:09.367318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587676-a06d-4bda-844e-72da64011b0b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:09.367797 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587676-a06d-4bda-844e-72da64011b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:09.368260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587676-a06d-4bda-844e-72da64011b0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:09.369151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77587676-a06d-4bda-844e-72da64011b0b None admin] RBAC: Adding query filter params (interface=public, service_id=de3aa3fb268743939bf24d94d9d9c030, region_id=RegionOne) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:09.369766 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77587676-a06d-4bda-844e-72da64011b0b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:09.371413 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77587676-a06d-4bda-844e-72da64011b0b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:09.377162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 351/701] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Oct 24 13:30:09 2022] GET /identity/v3/endpoints?service_id=de3aa3fb268743939bf24d94d9d9c030&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:09.382877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2229a-b96b-4a68-95d1-45ed22b9bb1f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:09.427885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587676-a06d-4bda-844e-72da64011b0b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:09.445823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:09.497314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2229a-b96b-4a68-95d1-45ed22b9bb1f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:09.537082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50731f8b-d3af-4091-9141-8a5566937262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:09.537700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50731f8b-d3af-4091-9141-8a5566937262 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:09.538192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50731f8b-d3af-4091-9141-8a5566937262 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:09.538393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50731f8b-d3af-4091-9141-8a5566937262 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:09.538983 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50731f8b-d3af-4091-9141-8a5566937262 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:09.540336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50731f8b-d3af-4091-9141-8a5566937262 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:09.545556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587676-a06d-4bda-844e-72da64011b0b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:09.552420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 351/702] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Oct 24 13:30:09 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:30:09.584739 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5deebd-9b87-4568-88d1-6b1ee3339c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:09.585757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5deebd-9b87-4568-88d1-6b1ee3339c83 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:09.586284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5deebd-9b87-4568-88d1-6b1ee3339c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:09.586755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5deebd-9b87-4568-88d1-6b1ee3339c83 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:09.587547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5deebd-9b87-4568-88d1-6b1ee3339c83 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:09.589163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5deebd-9b87-4568-88d1-6b1ee3339c83 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-8b5deebd-9b87-4568-88d1-6b1ee3339c83 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:09.599686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:09.600745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:09.601592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:09.604494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 352/703] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Oct 24 13:30:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:09.610876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50731f8b-d3af-4091-9141-8a5566937262 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:09.627962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:09.732287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50731f8b-d3af-4091-9141-8a5566937262 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:09.769802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e171a-f5b1-4896-88b4-37d408daf79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:09.771267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e171a-f5b1-4896-88b4-37d408daf79f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:09.771749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e171a-f5b1-4896-88b4-37d408daf79f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:09.772195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e171a-f5b1-4896-88b4-37d408daf79f None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:09.772995 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e171a-f5b1-4896-88b4-37d408daf79f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:09.773470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e171a-f5b1-4896-88b4-37d408daf79f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:09.775161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e171a-f5b1-4896-88b4-37d408daf79f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:09.783245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 352/704] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:30:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:09.791459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5deebd-9b87-4568-88d1-6b1ee3339c83 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:09.793045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d791-053a-428e-99bd-da87ad5c5d00 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:09.793486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d791-053a-428e-99bd-da87ad5c5d00 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:09.793908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d791-053a-428e-99bd-da87ad5c5d00 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:09.795505 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 353/705] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:09.800906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966311b-af03-488b-b5f7-8d270346a8e0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:09.801216 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966311b-af03-488b-b5f7-8d270346a8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:09.801489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966311b-af03-488b-b5f7-8d270346a8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:09.821859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b966311b-af03-488b-b5f7-8d270346a8e0 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:09.823200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b966311b-af03-488b-b5f7-8d270346a8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:09.892982 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:09.902214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 354/706] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:09.924188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5deebd-9b87-4568-88d1-6b1ee3339c83 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:09.961240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531185f6-52ac-422e-b24d-ca6f0acdaa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:09.962151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531185f6-52ac-422e-b24d-ca6f0acdaa2d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:09.962668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531185f6-52ac-422e-b24d-ca6f0acdaa2d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:09.963118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531185f6-52ac-422e-b24d-ca6f0acdaa2d None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:09.963918 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531185f6-52ac-422e-b24d-ca6f0acdaa2d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:09.964405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531185f6-52ac-422e-b24d-ca6f0acdaa2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:09.965935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531185f6-52ac-422e-b24d-ca6f0acdaa2d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:09.971498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 353/707] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:30:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:09.979287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966311b-af03-488b-b5f7-8d270346a8e0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:09.989436 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:10.087294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966311b-af03-488b-b5f7-8d270346a8e0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:10.127821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b8771-94f6-4b30-b0c3-37fa244893e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:10.128745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b8771-94f6-4b30-b0c3-37fa244893e5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:10.129234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b8771-94f6-4b30-b0c3-37fa244893e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:10.129918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b8771-94f6-4b30-b0c3-37fa244893e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:10.131006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b8771-94f6-4b30-b0c3-37fa244893e5 None admin] RBAC: Adding query filter params (interface=public, service_id=8a1cfcc4ede045b8afec683bca30b9c9, region_id=RegionOne) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:10.131495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b8771-94f6-4b30-b0c3-37fa244893e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:10.133041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b8771-94f6-4b30-b0c3-37fa244893e5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:10.139158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 355/708] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Oct 24 13:30:09 2022] GET /identity/v3/endpoints?service_id=8a1cfcc4ede045b8afec683bca30b9c9&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:10.147323 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531185f6-52ac-422e-b24d-ca6f0acdaa2d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:10.212515 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b8771-94f6-4b30-b0c3-37fa244893e5 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:10.251723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531185f6-52ac-422e-b24d-ca6f0acdaa2d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:10.289381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb1e54-e2fd-459a-bb5c-fa9bea8cfe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:10.290644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb1e54-e2fd-459a-bb5c-fa9bea8cfe9b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:10.291069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb1e54-e2fd-459a-bb5c-fa9bea8cfe9b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:10.291478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb1e54-e2fd-459a-bb5c-fa9bea8cfe9b None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:10.292017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:10.293629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb1e54-e2fd-459a-bb5c-fa9bea8cfe9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:10.295304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb1e54-e2fd-459a-bb5c-fa9bea8cfe9b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:10.310371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 354/709] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Oct 24 13:30:10 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:30:10.330286 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b8771-94f6-4b30-b0c3-37fa244893e5 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:10.371306 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e96fa7-a620-452f-8f3f-c76a39902453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:10.372040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e96fa7-a620-452f-8f3f-c76a39902453 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:10.372321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e96fa7-a620-452f-8f3f-c76a39902453 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:10.372592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e96fa7-a620-452f-8f3f-c76a39902453 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:10.373185 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e96fa7-a620-452f-8f3f-c76a39902453 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:10.374647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e96fa7-a620-452f-8f3f-c76a39902453 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b5e96fa7-a620-452f-8f3f-c76a39902453 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:10.384905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:10.386758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:10.387870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 356/710] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:30:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:10.392495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb1e54-e2fd-459a-bb5c-fa9bea8cfe9b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:10.402379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:10.407494 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106421cf-1ded-4039-ad4f-052d39e6421a None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:10.407800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106421cf-1ded-4039-ad4f-052d39e6421a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:10.408034 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106421cf-1ded-4039-ad4f-052d39e6421a None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:10.409650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 357/711] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:10.416894 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99851120-93c3-4b64-b7a2-2ecdd88e0bc9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:10.417212 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99851120-93c3-4b64-b7a2-2ecdd88e0bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:10.417431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99851120-93c3-4b64-b7a2-2ecdd88e0bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:10.438200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-99851120-93c3-4b64-b7a2-2ecdd88e0bc9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:10.439560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-99851120-93c3-4b64-b7a2-2ecdd88e0bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:10.515917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb1e54-e2fd-459a-bb5c-fa9bea8cfe9b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:10.535367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 358/712] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:10.556437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b48ac7-466c-4445-a242-9e340ece79a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:10.557319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b48ac7-466c-4445-a242-9e340ece79a9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:10.557758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b48ac7-466c-4445-a242-9e340ece79a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:10.558265 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b48ac7-466c-4445-a242-9e340ece79a9 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:10.559041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b48ac7-466c-4445-a242-9e340ece79a9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:10.559507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b48ac7-466c-4445-a242-9e340ece79a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:10.561008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b48ac7-466c-4445-a242-9e340ece79a9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:10.566269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 355/713] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:30:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:10.575489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99851120-93c3-4b64-b7a2-2ecdd88e0bc9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:10.645596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:10.685759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99851120-93c3-4b64-b7a2-2ecdd88e0bc9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:10.721520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850b4ea-f7e7-4d86-996c-fc4e3f8f2b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:10.722424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850b4ea-f7e7-4d86-996c-fc4e3f8f2b66 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:10.722856 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850b4ea-f7e7-4d86-996c-fc4e3f8f2b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:10.723848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850b4ea-f7e7-4d86-996c-fc4e3f8f2b66 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:10.724726 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850b4ea-f7e7-4d86-996c-fc4e3f8f2b66 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:10.725258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850b4ea-f7e7-4d86-996c-fc4e3f8f2b66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:10.726801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850b4ea-f7e7-4d86-996c-fc4e3f8f2b66 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:10.731863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 359/714] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:30:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:10.805781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b48ac7-466c-4445-a242-9e340ece79a9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:10.888160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:10.923294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b48ac7-466c-4445-a242-9e340ece79a9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:10.958927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b13534-68f4-4e96-b8db-976ec215a95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:10.959392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b13534-68f4-4e96-b8db-976ec215a95f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:10.959667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b13534-68f4-4e96-b8db-976ec215a95f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:10.959933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b13534-68f4-4e96-b8db-976ec215a95f None admin] PATH_INFO: `/v3/services/image` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:10.960512 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b13534-68f4-4e96-b8db-976ec215a95f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:10.961869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b13534-68f4-4e96-b8db-976ec215a95f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-a8b13534-68f4-4e96-b8db-976ec215a95f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:10.970975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:10.971945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:10.973255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 356/715] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Oct 24 13:30:10 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:10.978146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850b4ea-f7e7-4d86-996c-fc4e3f8f2b66 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:11.086679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850b4ea-f7e7-4d86-996c-fc4e3f8f2b66 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:11.115089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:11.123101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb81129-4da4-4fda-95b6-fab67ab7b6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:11.123655 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb81129-4da4-4fda-95b6-fab67ab7b6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:11.123939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb81129-4da4-4fda-95b6-fab67ab7b6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:11.124216 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb81129-4da4-4fda-95b6-fab67ab7b6c1 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:11.124857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb81129-4da4-4fda-95b6-fab67ab7b6c1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:11.125448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb81129-4da4-4fda-95b6-fab67ab7b6c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:11.126845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb81129-4da4-4fda-95b6-fab67ab7b6c1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:11.133120 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 360/716] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:11.139280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b13534-68f4-4e96-b8db-976ec215a95f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:11.260091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b13534-68f4-4e96-b8db-976ec215a95f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:11.298410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2236c0e-230b-44e0-8461-c64fc5ababf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:11.299023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236c0e-230b-44e0-8461-c64fc5ababf6 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:11.299302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236c0e-230b-44e0-8461-c64fc5ababf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:11.299582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236c0e-230b-44e0-8461-c64fc5ababf6 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:11.300216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2236c0e-230b-44e0-8461-c64fc5ababf6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:11.300517 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2236c0e-230b-44e0-8461-c64fc5ababf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:11.301909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2236c0e-230b-44e0-8461-c64fc5ababf6 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:11.303241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:11.307135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 357/717] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:11 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:11.947135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7426a48-b64e-4cea-86e2-5e3768d7018f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:11.947694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7426a48-b64e-4cea-86e2-5e3768d7018f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:11.947694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7426a48-b64e-4cea-86e2-5e3768d7018f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:11.949231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 361/718] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:11.954719 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e85649-645a-448b-afe7-f45eb4a4904f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:11.955160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e85649-645a-448b-afe7-f45eb4a4904f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:11.955251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e85649-645a-448b-afe7-f45eb4a4904f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:11.974021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-82e85649-645a-448b-afe7-f45eb4a4904f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:11.975225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-82e85649-645a-448b-afe7-f45eb4a4904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:12.043305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 358/719] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:12.315128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7426a48-b64e-4cea-86e2-5e3768d7018f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:12.421417 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7426a48-b64e-4cea-86e2-5e3768d7018f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:12.456222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d41bf-7a85-4642-80b0-54225b7361c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:12.457174 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d41bf-7a85-4642-80b0-54225b7361c1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:12.457809 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d41bf-7a85-4642-80b0-54225b7361c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:12.458314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d41bf-7a85-4642-80b0-54225b7361c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:12.459071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d41bf-7a85-4642-80b0-54225b7361c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:12.460656 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d41bf-7a85-4642-80b0-54225b7361c1 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-962d41bf-7a85-4642-80b0-54225b7361c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:12.470380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:12.471468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:30:12.472304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:12.475738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 362/720] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:30:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:12.483691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e85649-645a-448b-afe7-f45eb4a4904f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:12.588192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e85649-645a-448b-afe7-f45eb4a4904f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:12.597248 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4878c2c-8f63-44ff-ab84-e2cf54986d05 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:12.597662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4878c2c-8f63-44ff-ab84-e2cf54986d05 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:12.598123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4878c2c-8f63-44ff-ab84-e2cf54986d05 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:12.600942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 363/721] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:12.605095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76821966-e05c-485c-9e8e-878a7fad55c5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:12.605533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76821966-e05c-485c-9e8e-878a7fad55c5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:12.606030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76821966-e05c-485c-9e8e-878a7fad55c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:12.616868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:12.625752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3984e-5352-4844-89db-26029f0d0639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:12.626389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3984e-5352-4844-89db-26029f0d0639 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:12.626736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3984e-5352-4844-89db-26029f0d0639 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:12.627015 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3984e-5352-4844-89db-26029f0d0639 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:12.627651 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3984e-5352-4844-89db-26029f0d0639 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:12.627952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3984e-5352-4844-89db-26029f0d0639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:12.629864 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3984e-5352-4844-89db-26029f0d0639 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:12.630292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-76821966-e05c-485c-9e8e-878a7fad55c5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:12.631916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-76821966-e05c-485c-9e8e-878a7fad55c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:12.635339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 359/722] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:30:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:12.643127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3984e-5352-4844-89db-26029f0d0639 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:12.686782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:12.705654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 364/723] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:12.750626 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3984e-5352-4844-89db-26029f0d0639 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:12.789326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccf5b8-efb1-4845-a20f-fd7938a95209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:12.789941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccf5b8-efb1-4845-a20f-fd7938a95209 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:12.790261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccf5b8-efb1-4845-a20f-fd7938a95209 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:12.790788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccf5b8-efb1-4845-a20f-fd7938a95209 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:12.791413 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccf5b8-efb1-4845-a20f-fd7938a95209 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:12.791709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccf5b8-efb1-4845-a20f-fd7938a95209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:12.793166 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccf5b8-efb1-4845-a20f-fd7938a95209 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:12.798584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 360/724] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Oct 24 13:30:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:12.804101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76821966-e05c-485c-9e8e-878a7fad55c5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:12.909583 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76821966-e05c-485c-9e8e-878a7fad55c5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:12.967744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cad7b-8589-4cf9-a260-92f1ded66367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:12.981831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cad7b-8589-4cf9-a260-92f1ded66367 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:12.981831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cad7b-8589-4cf9-a260-92f1ded66367 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:12.981831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cad7b-8589-4cf9-a260-92f1ded66367 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:12.981831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251cad7b-8589-4cf9-a260-92f1ded66367 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:12.981831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251cad7b-8589-4cf9-a260-92f1ded66367 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:12.997215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccf5b8-efb1-4845-a20f-fd7938a95209 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:13.023629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 365/725] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:30:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:13.118665 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:13.138800 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccf5b8-efb1-4845-a20f-fd7938a95209 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:13.182840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf8fb6-15b6-49f1-8f89-1454befdd2c7 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:13.183193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf8fb6-15b6-49f1-8f89-1454befdd2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:13.183395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf8fb6-15b6-49f1-8f89-1454befdd2c7 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:13.184963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 366/726] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:13.192731 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e30ddc-43b2-45ee-868a-87660fc75c0f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:13.193040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e30ddc-43b2-45ee-868a-87660fc75c0f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:13.193301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e30ddc-43b2-45ee-868a-87660fc75c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:13.204259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a788598-5093-4433-859f-3b3420b8eb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:13.209160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a788598-5093-4433-859f-3b3420b8eb84 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:13.209160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a788598-5093-4433-859f-3b3420b8eb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:13.209160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a788598-5093-4433-859f-3b3420b8eb84 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:13.209160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a788598-5093-4433-859f-3b3420b8eb84 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:13.216266 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a788598-5093-4433-859f-3b3420b8eb84 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6a788598-5093-4433-859f-3b3420b8eb84 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:13.232304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:13.233522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:13.234500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:13.235309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 361/727] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Oct 24 13:30:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:13.237076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-17e30ddc-43b2-45ee-868a-87660fc75c0f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:13.238679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-17e30ddc-43b2-45ee-868a-87660fc75c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:13.242556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a788598-5093-4433-859f-3b3420b8eb84 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:13.293547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:13.345541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 367/728] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:13.396109 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a788598-5093-4433-859f-3b3420b8eb84 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:13.414715 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:13.434760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73ed53-fe6b-42ca-93d0-396a7eaa0f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:13.435365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73ed53-fe6b-42ca-93d0-396a7eaa0f1c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:13.435641 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73ed53-fe6b-42ca-93d0-396a7eaa0f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:13.436723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73ed53-fe6b-42ca-93d0-396a7eaa0f1c None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:13.437366 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73ed53-fe6b-42ca-93d0-396a7eaa0f1c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:13.437669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73ed53-fe6b-42ca-93d0-396a7eaa0f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:13.439149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73ed53-fe6b-42ca-93d0-396a7eaa0f1c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:13.444756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 362/729] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:30:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:13.450723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e30ddc-43b2-45ee-868a-87660fc75c0f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:13.581269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e30ddc-43b2-45ee-868a-87660fc75c0f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:13.610917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197c16d-e3a8-4cc8-b5fc-632c99f2d5da None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:13.611244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197c16d-e3a8-4cc8-b5fc-632c99f2d5da None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:13.611470 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197c16d-e3a8-4cc8-b5fc-632c99f2d5da None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:13.612944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 363/730] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:13.620862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d970f-d8e7-4423-a2a9-235e4ce1631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:13.621490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d970f-d8e7-4423-a2a9-235e4ce1631e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:13.621769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d970f-d8e7-4423-a2a9-235e4ce1631e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:13.622040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d970f-d8e7-4423-a2a9-235e4ce1631e None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:13.622701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d970f-d8e7-4423-a2a9-235e4ce1631e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:13.623003 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d970f-d8e7-4423-a2a9-235e4ce1631e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:13.623602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada6cc1-0f13-4a38-9522-ca2b98597a97 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:13.623892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada6cc1-0f13-4a38-9522-ca2b98597a97 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:13.626281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada6cc1-0f13-4a38-9522-ca2b98597a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:13.626416 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d970f-d8e7-4423-a2a9-235e4ce1631e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:13.634285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 368/731] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:30:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:13.638133 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d970f-d8e7-4423-a2a9-235e4ce1631e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:13.662649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1ada6cc1-0f13-4a38-9522-ca2b98597a97 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:13.665585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1ada6cc1-0f13-4a38-9522-ca2b98597a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:13.689146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:13.752810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 364/732] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:13.753662 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada6cc1-0f13-4a38-9522-ca2b98597a97 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:13.760958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d970f-d8e7-4423-a2a9-235e4ce1631e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:13.798578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7989076-4eae-4eef-a4c8-4bd452b77be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:13.799557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7989076-4eae-4eef-a4c8-4bd452b77be9 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:13.800039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7989076-4eae-4eef-a4c8-4bd452b77be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:13.800520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7989076-4eae-4eef-a4c8-4bd452b77be9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:13.801966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7989076-4eae-4eef-a4c8-4bd452b77be9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:13.803735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7989076-4eae-4eef-a4c8-4bd452b77be9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:13.830762 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 369/733] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:30:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:13.878650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada6cc1-0f13-4a38-9522-ca2b98597a97 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:13.892387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:13.931669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2c082-e488-4a9c-b861-cafa22c19494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:13.932352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c082-e488-4a9c-b861-cafa22c19494 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:13.932675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c082-e488-4a9c-b861-cafa22c19494 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:13.932984 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c082-e488-4a9c-b861-cafa22c19494 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:13.933694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2c082-e488-4a9c-b861-cafa22c19494 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:13.935184 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2c082-e488-4a9c-b861-cafa22c19494 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:13.954000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 365/734] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:30:13 2022] POST /identity/v3/services => generated 242 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:14.028748 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7989076-4eae-4eef-a4c8-4bd452b77be9 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:14.141723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:14.153714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7989076-4eae-4eef-a4c8-4bd452b77be9 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:14.203990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f21ab1-7a84-46d1-8c5e-42afd22fc2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:14.204616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f21ab1-7a84-46d1-8c5e-42afd22fc2ba None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:14.204893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f21ab1-7a84-46d1-8c5e-42afd22fc2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:14.205166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f21ab1-7a84-46d1-8c5e-42afd22fc2ba None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:14.205888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f21ab1-7a84-46d1-8c5e-42afd22fc2ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:14.207457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f21ab1-7a84-46d1-8c5e-42afd22fc2ba None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:14.228811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 370/735] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:30:14 2022] POST /identity/v3/services => generated 242 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:15.419193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d65305-7cdf-4124-9192-2d88ec03fa54 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.419731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d65305-7cdf-4124-9192-2d88ec03fa54 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.419731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d65305-7cdf-4124-9192-2d88ec03fa54 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.421312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 366/736] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:15.427330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f75e8-bbba-4f17-9939-76b7d492cb5b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.427730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f75e8-bbba-4f17-9939-76b7d492cb5b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.427926 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f75e8-bbba-4f17-9939-76b7d492cb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.446921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-425f75e8-bbba-4f17-9939-76b7d492cb5b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:15.448103 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-425f75e8-bbba-4f17-9939-76b7d492cb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:15.530361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 371/737] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:15 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:15.794002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d65305-7cdf-4124-9192-2d88ec03fa54 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:15.907018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d65305-7cdf-4124-9192-2d88ec03fa54 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:15.942917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec49482-75cd-48b4-9381-ad8b6aa14519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:15.943532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec49482-75cd-48b4-9381-ad8b6aa14519 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.943804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec49482-75cd-48b4-9381-ad8b6aa14519 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.944070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec49482-75cd-48b4-9381-ad8b6aa14519 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.944732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec49482-75cd-48b4-9381-ad8b6aa14519 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:15.946150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec49482-75cd-48b4-9381-ad8b6aa14519 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:15.948793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WARNING keystone.server.flask.common [None req-2ec49482-75cd-48b4-9381-ad8b6aa14519 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 24 13:30:15.968378 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.sql.core [None req-2ec49482-75cd-48b4-9381-ad8b6aa14519 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 13:30:15.968378 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:30:15.968378 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [parameters: {'id': '7ffa0f797f174de8a38698df892dbb46', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 24 13:30:15.968378 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81688) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-2ec49482-75cd-48b4-9381-ad8b6aa14519 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7ffa0f797f174de8a38698df892dbb46.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7ffa0f797f174de8a38698df892dbb46. Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:30:15.976420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result.read() Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 13:30:15.977241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application yield resource Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise self.value Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application session.commit() Oct 24 13:30:15.978210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:30:15.979886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:30:15.980989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result.read() Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:30:15.981959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [parameters: {'id': '7ffa0f797f174de8a38698df892dbb46', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:15.984098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:15.986285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:15.990978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:15.990978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 24 13:30:15.990978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 24 13:30:15.990978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:15.990978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:15.990978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 13:30:15.990978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 13:30:15.990978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7ffa0f797f174de8a38698df892dbb46. Oct 24 13:30:15.990978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:15.990978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 367/738] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Oct 24 13:30:15 2022] POST /identity/v3/projects => generated 262 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:15.991547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f75e8-bbba-4f17-9939-76b7d492cb5b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:16.074410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:16.104556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f75e8-bbba-4f17-9939-76b7d492cb5b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:16.146244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae556d7f-35a5-4527-a8db-397d67db41d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:16.147268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556d7f-35a5-4527-a8db-397d67db41d2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:16.147546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556d7f-35a5-4527-a8db-397d67db41d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:16.147820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae556d7f-35a5-4527-a8db-397d67db41d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:16.155178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae556d7f-35a5-4527-a8db-397d67db41d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:16.155521 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ebbbd-50b0-4636-868b-206623dfef8e None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:16.156007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ebbbd-50b0-4636-868b-206623dfef8e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:16.156514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ebbbd-50b0-4636-868b-206623dfef8e None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:16.156628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae556d7f-35a5-4527-a8db-397d67db41d2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:16.166234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 368/739] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:30:16.167692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58fc55-c6a9-4291-9d6a-09bf5adcdf74 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:16.168188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58fc55-c6a9-4291-9d6a-09bf5adcdf74 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:16.168667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58fc55-c6a9-4291-9d6a-09bf5adcdf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-ae556d7f-35a5-4527-a8db-397d67db41d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:30:16.169002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:16.169912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:16.170820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:30:16.170820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:30:16.170820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:30:16.170820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:30:16.170820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:30:16.170820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:30:16.170820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:30:16.170820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:16.170820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 372/740] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:30:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:16.179244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae556d7f-35a5-4527-a8db-397d67db41d2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:16.193138 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ba58fc55-c6a9-4291-9d6a-09bf5adcdf74 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:16.195234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ba58fc55-c6a9-4291-9d6a-09bf5adcdf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:16.224653 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:16.283639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 369/741] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:16 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:16.286571 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2fe6-0ca2-4e28-b16e-007eb4533d56 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:16.287263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2fe6-0ca2-4e28-b16e-007eb4533d56 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:16.287738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2fe6-0ca2-4e28-b16e-007eb4533d56 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:16.288256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae556d7f-35a5-4527-a8db-397d67db41d2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:16.289702 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 370/742] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:16.295186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a8872-9627-4eb7-844b-675f9d86e0ca None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:16.295467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a8872-9627-4eb7-844b-675f9d86e0ca None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:16.295894 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a8872-9627-4eb7-844b-675f9d86e0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:16.318251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b16a8872-9627-4eb7-844b-675f9d86e0ca None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:16.319629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b16a8872-9627-4eb7-844b-675f9d86e0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:16.339724 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748f806-bf8b-4d0a-90c0-db67735c5bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:16.340748 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748f806-bf8b-4d0a-90c0-db67735c5bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:16.341208 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748f806-bf8b-4d0a-90c0-db67735c5bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:16.341766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748f806-bf8b-4d0a-90c0-db67735c5bb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:16.342625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748f806-bf8b-4d0a-90c0-db67735c5bb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:16.343118 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748f806-bf8b-4d0a-90c0-db67735c5bb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:16.344943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748f806-bf8b-4d0a-90c0-db67735c5bb9 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:16.346740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:16.365732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 373/743] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:30:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:16.399586 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 371/744] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:16 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:16.426317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8113d-4923-4f1b-bb94-b03fcabfe517 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:16.427203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8113d-4923-4f1b-bb94-b03fcabfe517 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:16.427719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8113d-4923-4f1b-bb94-b03fcabfe517 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:16.430136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 374/745] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:16.435497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482be795-e8c5-495d-ba37-ee4eab907403 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:16.436380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482be795-e8c5-495d-ba37-ee4eab907403 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:16.436891 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482be795-e8c5-495d-ba37-ee4eab907403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:16.456880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-482be795-e8c5-495d-ba37-ee4eab907403 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:16.459098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-482be795-e8c5-495d-ba37-ee4eab907403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:16.538615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 372/746] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:16 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:16.591337 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8113d-4923-4f1b-bb94-b03fcabfe517 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:16.690287 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482be795-e8c5-495d-ba37-ee4eab907403 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:16.702211 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8113d-4923-4f1b-bb94-b03fcabfe517 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:16.705891 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:16.743194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007954e3-6d9c-4698-91d2-d65b90e982fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:16.744216 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007954e3-6d9c-4698-91d2-d65b90e982fe None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:16.744710 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007954e3-6d9c-4698-91d2-d65b90e982fe None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:16.744983 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007954e3-6d9c-4698-91d2-d65b90e982fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:16.745564 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007954e3-6d9c-4698-91d2-d65b90e982fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:16.746954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007954e3-6d9c-4698-91d2-d65b90e982fe None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-007954e3-6d9c-4698-91d2-d65b90e982fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:16.756936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:16.757947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:16.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 375/747] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Oct 24 13:30:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:16.764172 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007954e3-6d9c-4698-91d2-d65b90e982fe None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:16.813852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482be795-e8c5-495d-ba37-ee4eab907403 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:16.831898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:16.853640 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c2541-54d3-497b-ac65-416cc5e116df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:16.854214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c2541-54d3-497b-ac65-416cc5e116df None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:16.854492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c2541-54d3-497b-ac65-416cc5e116df None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:16.854759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c2541-54d3-497b-ac65-416cc5e116df None admin] PATH_INFO: `/v3/services/compute` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:16.855330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c2541-54d3-497b-ac65-416cc5e116df None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:16.856798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c2541-54d3-497b-ac65-416cc5e116df None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-f02c2541-54d3-497b-ac65-416cc5e116df None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:16.866248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:16.867218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:16.868051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 373/748] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:30:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:16.868917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c2541-54d3-497b-ac65-416cc5e116df None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:16.885144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007954e3-6d9c-4698-91d2-d65b90e982fe None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:16.921910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a339a-f8fe-4518-b766-9cec6fbb4029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:16.922545 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a339a-f8fe-4518-b766-9cec6fbb4029 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:16.922823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a339a-f8fe-4518-b766-9cec6fbb4029 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:16.923091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a339a-f8fe-4518-b766-9cec6fbb4029 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:16.923706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a339a-f8fe-4518-b766-9cec6fbb4029 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:16.924003 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a339a-f8fe-4518-b766-9cec6fbb4029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:16.925373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a339a-f8fe-4518-b766-9cec6fbb4029 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:16.931379 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 376/749] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:30:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:16.931956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a339a-f8fe-4518-b766-9cec6fbb4029 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:16.957940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:16.984853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c2541-54d3-497b-ac65-416cc5e116df None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.021736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3b028-3b94-4058-aa27-7939b31fca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.022395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3b028-3b94-4058-aa27-7939b31fca17 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.022664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3b028-3b94-4058-aa27-7939b31fca17 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.022932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3b028-3b94-4058-aa27-7939b31fca17 None admin] PATH_INFO: `/v3/services/image` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.023505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3b028-3b94-4058-aa27-7939b31fca17 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.024896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3b028-3b94-4058-aa27-7939b31fca17 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-e7c3b028-3b94-4058-aa27-7939b31fca17 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:17.033732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:17.034883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:17.036124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 374/750] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Oct 24 13:30:16 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:17.037126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3b028-3b94-4058-aa27-7939b31fca17 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:17.045795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a339a-f8fe-4518-b766-9cec6fbb4029 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.072589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:17.084657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d12c7a-51e4-414b-b9e8-f7ccd94c74c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.085266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12c7a-51e4-414b-b9e8-f7ccd94c74c4 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.085988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12c7a-51e4-414b-b9e8-f7ccd94c74c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.086302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12c7a-51e4-414b-b9e8-f7ccd94c74c4 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.086936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d12c7a-51e4-414b-b9e8-f7ccd94c74c4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:17.087240 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d12c7a-51e4-414b-b9e8-f7ccd94c74c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.089298 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d12c7a-51e4-414b-b9e8-f7ccd94c74c4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.094488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 377/751] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:30:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:17.099982 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d12c7a-51e4-414b-b9e8-f7ccd94c74c4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:17.148195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3b028-3b94-4058-aa27-7939b31fca17 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.184414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c1ba0-1333-4335-b714-e798c12cdaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.185022 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c1ba0-1333-4335-b714-e798c12cdaae None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.185344 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c1ba0-1333-4335-b714-e798c12cdaae None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.185616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c1ba0-1333-4335-b714-e798c12cdaae None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.186272 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c1ba0-1333-4335-b714-e798c12cdaae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:17.186554 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c1ba0-1333-4335-b714-e798c12cdaae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.187901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c1ba0-1333-4335-b714-e798c12cdaae None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.198133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 375/752] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:30:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:17.198133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c1ba0-1333-4335-b714-e798c12cdaae None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:17.199292 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:17.214075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d12c7a-51e4-414b-b9e8-f7ccd94c74c4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.249935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ead3e-eb8e-478d-9038-8d06e49d9d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.250590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ead3e-eb8e-478d-9038-8d06e49d9d19 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.250871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ead3e-eb8e-478d-9038-8d06e49d9d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.251139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ead3e-eb8e-478d-9038-8d06e49d9d19 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.251779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ead3e-eb8e-478d-9038-8d06e49d9d19 None admin] RBAC: Adding query filter params (name=image) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:17.252081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ead3e-eb8e-478d-9038-8d06e49d9d19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.253428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ead3e-eb8e-478d-9038-8d06e49d9d19 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.258967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 378/753] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:17.259749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ead3e-eb8e-478d-9038-8d06e49d9d19 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:17.318756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c1ba0-1333-4335-b714-e798c12cdaae None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.319974 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:17.358094 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0799d-6288-447d-ae80-398248cb22f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.358752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0799d-6288-447d-ae80-398248cb22f2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.359045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0799d-6288-447d-ae80-398248cb22f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.359302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0799d-6288-447d-ae80-398248cb22f2 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.359945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0799d-6288-447d-ae80-398248cb22f2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:17.360248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0799d-6288-447d-ae80-398248cb22f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.361641 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0799d-6288-447d-ae80-398248cb22f2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.370216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 376/754] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:30:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:17.370216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0799d-6288-447d-ae80-398248cb22f2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:17.384142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ead3e-eb8e-478d-9038-8d06e49d9d19 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.425984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca56bc9-6f5a-4ab2-8599-d1cd1fad61f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.426511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca56bc9-6f5a-4ab2-8599-d1cd1fad61f7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.427074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca56bc9-6f5a-4ab2-8599-d1cd1fad61f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.427347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca56bc9-6f5a-4ab2-8599-d1cd1fad61f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.428292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca56bc9-6f5a-4ab2-8599-d1cd1fad61f7 None admin] RBAC: Adding query filter params (interface=admin, service_id=8a1cfcc4ede045b8afec683bca30b9c9, region_id=RegionOne) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:17.428827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca56bc9-6f5a-4ab2-8599-d1cd1fad61f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.430645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca56bc9-6f5a-4ab2-8599-d1cd1fad61f7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.436595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 379/755] 10.222.0.20 () {56 vars in 1249 bytes} [Mon Oct 24 13:30:17 2022] GET /identity/v3/endpoints?service_id=8a1cfcc4ede045b8afec683bca30b9c9&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:17.437450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca56bc9-6f5a-4ab2-8599-d1cd1fad61f7 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:17.443030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:17.481135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0799d-6288-447d-ae80-398248cb22f2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.516576 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343bcf9-26f9-4a82-9841-e959a8c0fdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.517193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343bcf9-26f9-4a82-9841-e959a8c0fdfc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.517446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343bcf9-26f9-4a82-9841-e959a8c0fdfc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.517724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343bcf9-26f9-4a82-9841-e959a8c0fdfc None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.518352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343bcf9-26f9-4a82-9841-e959a8c0fdfc None admin] RBAC: Adding query filter params (type=image) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:17.518650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343bcf9-26f9-4a82-9841-e959a8c0fdfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.520513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343bcf9-26f9-4a82-9841-e959a8c0fdfc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.530239 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 377/756] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:17.530916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343bcf9-26f9-4a82-9841-e959a8c0fdfc None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:17.542193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca56bc9-6f5a-4ab2-8599-d1cd1fad61f7 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.556711 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:17.582123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33002e0-414f-4d39-a0cc-17dd51005b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.583610 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33002e0-414f-4d39-a0cc-17dd51005b08 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.583975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33002e0-414f-4d39-a0cc-17dd51005b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.584246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33002e0-414f-4d39-a0cc-17dd51005b08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.584937 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33002e0-414f-4d39-a0cc-17dd51005b08 None admin] RBAC: Adding query filter params (interface=public, service_id=338cf26581ce4e66bdc438129bea38c0, region_id=RegionOne) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:17.585241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33002e0-414f-4d39-a0cc-17dd51005b08 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.586625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33002e0-414f-4d39-a0cc-17dd51005b08 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.592102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 380/757] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Oct 24 13:30:17 2022] GET /identity/v3/endpoints?service_id=338cf26581ce4e66bdc438129bea38c0&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:17.597317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33002e0-414f-4d39-a0cc-17dd51005b08 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:17.642395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343bcf9-26f9-4a82-9841-e959a8c0fdfc None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.674471 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:17.680335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc5362-c614-4c74-ab1b-2ae2a5dd7a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.680852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc5362-c614-4c74-ab1b-2ae2a5dd7a49 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.681129 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc5362-c614-4c74-ab1b-2ae2a5dd7a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.681395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc5362-c614-4c74-ab1b-2ae2a5dd7a49 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.681990 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc5362-c614-4c74-ab1b-2ae2a5dd7a49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.683364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc5362-c614-4c74-ab1b-2ae2a5dd7a49 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.690089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 378/758] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Oct 24 13:30:17 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:30:17.690711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc5362-c614-4c74-ab1b-2ae2a5dd7a49 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:17.721016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33002e0-414f-4d39-a0cc-17dd51005b08 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.775307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2b296-cc0a-4d21-bd9b-9f7252cd15fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.777517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2b296-cc0a-4d21-bd9b-9f7252cd15fb None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.778057 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2b296-cc0a-4d21-bd9b-9f7252cd15fb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.778560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2b296-cc0a-4d21-bd9b-9f7252cd15fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.779523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2b296-cc0a-4d21-bd9b-9f7252cd15fb None admin] RBAC: Adding query filter params (interface=public, service_id=fdc7f189a9014b4fb116b584f5da4a63, region_id=RegionOne) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:17.780070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2b296-cc0a-4d21-bd9b-9f7252cd15fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.781690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2b296-cc0a-4d21-bd9b-9f7252cd15fb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.790222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 381/759] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Oct 24 13:30:17 2022] GET /identity/v3/endpoints?service_id=fdc7f189a9014b4fb116b584f5da4a63&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:17.799316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2b296-cc0a-4d21-bd9b-9f7252cd15fb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:17.813257 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc5362-c614-4c74-ab1b-2ae2a5dd7a49 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.859815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da84a0a-9a6e-4c7f-bbb1-343c9e602923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.860523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da84a0a-9a6e-4c7f-bbb1-343c9e602923 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.860878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da84a0a-9a6e-4c7f-bbb1-343c9e602923 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.861180 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da84a0a-9a6e-4c7f-bbb1-343c9e602923 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.861865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da84a0a-9a6e-4c7f-bbb1-343c9e602923 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.863385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da84a0a-9a6e-4c7f-bbb1-343c9e602923 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.871308 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 379/760] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Oct 24 13:30:17 2022] GET /identity/v3/services => generated 1957 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:30:17.871509 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:17.935698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2b296-cc0a-4d21-bd9b-9f7252cd15fb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.976373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00323c4-7040-4ae5-b099-7962bb1c649a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.976998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00323c4-7040-4ae5-b099-7962bb1c649a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.977275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00323c4-7040-4ae5-b099-7962bb1c649a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.977540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00323c4-7040-4ae5-b099-7962bb1c649a None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.978153 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00323c4-7040-4ae5-b099-7962bb1c649a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.979640 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00323c4-7040-4ae5-b099-7962bb1c649a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.987577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 382/761] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Oct 24 13:30:17 2022] GET /identity/v3/services => generated 1957 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:30:18.655536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470f23f-d1d4-45b4-ab10-b919c3af5ccd None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:18.656077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470f23f-d1d4-45b4-ab10-b919c3af5ccd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:18.656077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470f23f-d1d4-45b4-ab10-b919c3af5ccd None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:18.657466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 380/762] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:18.664349 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46536115-ba7b-40f3-bedb-ea575450e7ce None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:18.664872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46536115-ba7b-40f3-bedb-ea575450e7ce None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:18.664872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46536115-ba7b-40f3-bedb-ea575450e7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:18.684957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-46536115-ba7b-40f3-bedb-ea575450e7ce None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:18.686131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-46536115-ba7b-40f3-bedb-ea575450e7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:18.777420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 383/763] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:18 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:19.096093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470f23f-d1d4-45b4-ab10-b919c3af5ccd None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:19.158203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:19.211665 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470f23f-d1d4-45b4-ab10-b919c3af5ccd None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:19.248981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5626-ed31-47f4-acaf-47c1a95879f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:19.249504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5626-ed31-47f4-acaf-47c1a95879f5 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:19.249779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5626-ed31-47f4-acaf-47c1a95879f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:19.250043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5626-ed31-47f4-acaf-47c1a95879f5 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:19.250716 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5626-ed31-47f4-acaf-47c1a95879f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:19.252176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5626-ed31-47f4-acaf-47c1a95879f5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:19.267876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.sql.core [None req-965b5626-ed31-47f4-acaf-47c1a95879f5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 13:30:19.267876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 24 13:30:19.267876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [parameters: {'user_id': '4de955d682b64c22828849da88bdeabb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 24 13:30:19.267876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81688) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-965b5626-ed31-47f4-acaf-47c1a95879f5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result.read() Oct 24 13:30:19.275772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:30:19.276645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application yield resource Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise self.value Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application session.commit() Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:30:19.278282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:30:19.280324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:30:19.287186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result.read() Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:30:19.289126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [parameters: {'user_id': '4de955d682b64c22828849da88bdeabb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:19.291320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 24 13:30:19.293123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 24 13:30:19.295326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 24 13:30:19.295326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 24 13:30:19.295326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:30:19.295326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:30:19.295326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 24 13:30:19.295326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:19.295326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 381/764] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Oct 24 13:30:19 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:19.296217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46536115-ba7b-40f3-bedb-ea575450e7ce None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:19.351201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:19.440251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46536115-ba7b-40f3-bedb-ea575450e7ce None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:19.480869 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98754b4d-aa7a-4c42-8e19-05a8c4823dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:19.481865 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98754b4d-aa7a-4c42-8e19-05a8c4823dda None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:19.482406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98754b4d-aa7a-4c42-8e19-05a8c4823dda None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:19.482954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98754b4d-aa7a-4c42-8e19-05a8c4823dda None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:19.491546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98754b4d-aa7a-4c42-8e19-05a8c4823dda None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:19.493216 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98754b4d-aa7a-4c42-8e19-05a8c4823dda None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-98754b4d-aa7a-4c42-8e19-05a8c4823dda None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:30:19.505203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:19.506203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:19.507168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:19.507168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:19.507168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:30:19.507168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:30:19.507168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:30:19.507168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:30:19.507168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:30:19.507168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:30:19.507168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:30:19.507168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:19.509826 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 384/765] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:30:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:19.522114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5626-ed31-47f4-acaf-47c1a95879f5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:19.683185 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5626-ed31-47f4-acaf-47c1a95879f5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:19.708847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:19.722112 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c48992-50ce-451e-b88a-272706a717dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:19.723295 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c48992-50ce-451e-b88a-272706a717dc None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:19.724071 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c48992-50ce-451e-b88a-272706a717dc None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:19.724488 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c48992-50ce-451e-b88a-272706a717dc None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:19.725347 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c48992-50ce-451e-b88a-272706a717dc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:19.726581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c48992-50ce-451e-b88a-272706a717dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:19.728356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c48992-50ce-451e-b88a-272706a717dc None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:19.770705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 382/766] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Oct 24 13:30:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:20.161075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999260dd-2da3-4600-b65c-2023f525e9a4 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:20.161597 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999260dd-2da3-4600-b65c-2023f525e9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:20.161597 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999260dd-2da3-4600-b65c-2023f525e9a4 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:20.166213 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 385/767] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:20.168704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847b6af-9374-45ca-a00b-07ad725d6813 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:20.168987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847b6af-9374-45ca-a00b-07ad725d6813 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:20.169255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847b6af-9374-45ca-a00b-07ad725d6813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:20.187993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f847b6af-9374-45ca-a00b-07ad725d6813 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:20.189468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f847b6af-9374-45ca-a00b-07ad725d6813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:20.216787 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797758d7-09b5-410d-916b-f75a5c6bf681 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:20.217151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797758d7-09b5-410d-916b-f75a5c6bf681 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:20.217527 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797758d7-09b5-410d-916b-f75a5c6bf681 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:20.219128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 386/768] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:20.224375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03de701-6fd4-4757-8fa2-e90f60aea88e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:20.224657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03de701-6fd4-4757-8fa2-e90f60aea88e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:20.225090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03de701-6fd4-4757-8fa2-e90f60aea88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:20.244654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a03de701-6fd4-4757-8fa2-e90f60aea88e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:20.245860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a03de701-6fd4-4757-8fa2-e90f60aea88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:20.273828 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 383/769] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:20 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:20.320948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 387/770] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:20 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:20.372138 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde0eb7-a15f-4cd0-b8a9-ab70a973a6f1 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:20.372439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde0eb7-a15f-4cd0-b8a9-ab70a973a6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:20.372690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde0eb7-a15f-4cd0-b8a9-ab70a973a6f1 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:20.374047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 384/771] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:20.379433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a6c37-ee05-4751-9325-68e42ef97dab None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:20.379808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a6c37-ee05-4751-9325-68e42ef97dab None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:20.380083 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a6c37-ee05-4751-9325-68e42ef97dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:20.398469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7c9a6c37-ee05-4751-9325-68e42ef97dab None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:20.399639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7c9a6c37-ee05-4751-9325-68e42ef97dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:20.420512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:20.467623 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 388/772] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:20 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:20.547113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde0eb7-a15f-4cd0-b8a9-ab70a973a6f1 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:20.590375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a6c37-ee05-4751-9325-68e42ef97dab None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:20.659396 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde0eb7-a15f-4cd0-b8a9-ab70a973a6f1 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:20.672915 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:20.698464 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8afa9c-570a-4d4f-ac0f-f0318d1ce350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:20.699087 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8afa9c-570a-4d4f-ac0f-f0318d1ce350 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:20.699371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8afa9c-570a-4d4f-ac0f-f0318d1ce350 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:20.699644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8afa9c-570a-4d4f-ac0f-f0318d1ce350 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:20.700254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8afa9c-570a-4d4f-ac0f-f0318d1ce350 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:20.701616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8afa9c-570a-4d4f-ac0f-f0318d1ce350 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:20.706545 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a6c37-ee05-4751-9325-68e42ef97dab None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-2d8afa9c-570a-4d4f-ac0f-f0318d1ce350 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:20.711074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:20.712066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:30:20.712905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:20.717200 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 385/773] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Oct 24 13:30:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:20.718508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8afa9c-570a-4d4f-ac0f-f0318d1ce350 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:20.744050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20142a-e011-4284-ba23-a7d19b387def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:20.744692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20142a-e011-4284-ba23-a7d19b387def None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:20.744975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20142a-e011-4284-ba23-a7d19b387def None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:20.745249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20142a-e011-4284-ba23-a7d19b387def None admin] PATH_INFO: `/v3/services/compute` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:20.745829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20142a-e011-4284-ba23-a7d19b387def None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:20.747226 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20142a-e011-4284-ba23-a7d19b387def None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-5f20142a-e011-4284-ba23-a7d19b387def None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:20.756795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:20.757755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:20.758979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 389/774] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:30:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:20.759797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20142a-e011-4284-ba23-a7d19b387def None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:20.781810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:20.827698 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8afa9c-570a-4d4f-ac0f-f0318d1ce350 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:20.861292 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86773041-e16b-4e93-8f24-94e97fb0f8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:20.861894 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86773041-e16b-4e93-8f24-94e97fb0f8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:20.862196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86773041-e16b-4e93-8f24-94e97fb0f8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:20.862466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86773041-e16b-4e93-8f24-94e97fb0f8c1 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:20.862909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20142a-e011-4284-ba23-a7d19b387def None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:20.863089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86773041-e16b-4e93-8f24-94e97fb0f8c1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:20.863414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86773041-e16b-4e93-8f24-94e97fb0f8c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:20.864848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86773041-e16b-4e93-8f24-94e97fb0f8c1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:20.869652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 386/775] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:30:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:20.870437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86773041-e16b-4e93-8f24-94e97fb0f8c1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:20.898998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce234b3-580c-4d2b-9926-e4f006b5232d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:20.899927 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce234b3-580c-4d2b-9926-e4f006b5232d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:20.900392 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce234b3-580c-4d2b-9926-e4f006b5232d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:20.900690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce234b3-580c-4d2b-9926-e4f006b5232d None admin] PATH_INFO: `/v3/services/image` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:20.901553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce234b3-580c-4d2b-9926-e4f006b5232d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:20.902948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce234b3-580c-4d2b-9926-e4f006b5232d None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:20.911612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-2ce234b3-580c-4d2b-9926-e4f006b5232d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:20.911744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:20.912684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:20.913538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 390/776] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Oct 24 13:30:20 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:20.914545 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce234b3-580c-4d2b-9926-e4f006b5232d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:20.973869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86773041-e16b-4e93-8f24-94e97fb0f8c1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:20.997713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:21.009216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbbbb5-ba4d-464f-a8b1-2b2676ca4c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:21.009825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbbbb5-ba4d-464f-a8b1-2b2676ca4c33 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:21.010105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbbbb5-ba4d-464f-a8b1-2b2676ca4c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:21.010408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbbbb5-ba4d-464f-a8b1-2b2676ca4c33 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:21.011040 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbbbb5-ba4d-464f-a8b1-2b2676ca4c33 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:21.011341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbbbb5-ba4d-464f-a8b1-2b2676ca4c33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:21.012709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbbbb5-ba4d-464f-a8b1-2b2676ca4c33 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:21.014453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce234b3-580c-4d2b-9926-e4f006b5232d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:21.018476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 387/777] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:30:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:21.019260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbbbb5-ba4d-464f-a8b1-2b2676ca4c33 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:21.048309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a478b1-33d2-4ca4-b8d7-fab8f2033c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:21.049228 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a478b1-33d2-4ca4-b8d7-fab8f2033c05 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:21.049506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a478b1-33d2-4ca4-b8d7-fab8f2033c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:21.049778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a478b1-33d2-4ca4-b8d7-fab8f2033c05 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:21.050446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a478b1-33d2-4ca4-b8d7-fab8f2033c05 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:21.050747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a478b1-33d2-4ca4-b8d7-fab8f2033c05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:21.052712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a478b1-33d2-4ca4-b8d7-fab8f2033c05 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:21.057639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 391/778] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Oct 24 13:30:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:21.058433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a478b1-33d2-4ca4-b8d7-fab8f2033c05 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:21.122975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbbbb5-ba4d-464f-a8b1-2b2676ca4c33 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:21.124945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:21.157451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a478b1-33d2-4ca4-b8d7-fab8f2033c05 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:21.157754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5bb0c-4e07-4c15-abff-244cf6c1367a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:21.158453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5bb0c-4e07-4c15-abff-244cf6c1367a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:21.158788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5bb0c-4e07-4c15-abff-244cf6c1367a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:21.159121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5bb0c-4e07-4c15-abff-244cf6c1367a None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:21.159808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5bb0c-4e07-4c15-abff-244cf6c1367a None admin] RBAC: Adding query filter params (name=image) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:21.160176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5bb0c-4e07-4c15-abff-244cf6c1367a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:21.162263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5bb0c-4e07-4c15-abff-244cf6c1367a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:21.167293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 388/779] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:21.168033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5bb0c-4e07-4c15-abff-244cf6c1367a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:21.198264 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345003aa-51ca-46d3-bc6f-6736e3fd0ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:21.198900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345003aa-51ca-46d3-bc6f-6736e3fd0ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:21.199144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345003aa-51ca-46d3-bc6f-6736e3fd0ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:21.199416 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345003aa-51ca-46d3-bc6f-6736e3fd0ac6 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:21.200039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345003aa-51ca-46d3-bc6f-6736e3fd0ac6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:21.200340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345003aa-51ca-46d3-bc6f-6736e3fd0ac6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:21.201699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345003aa-51ca-46d3-bc6f-6736e3fd0ac6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:21.206707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 392/780] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Oct 24 13:30:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:21.207932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345003aa-51ca-46d3-bc6f-6736e3fd0ac6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:21.225852 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:21.275459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5bb0c-4e07-4c15-abff-244cf6c1367a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:21.308993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95b480-c2ab-4a4f-b6e3-9699a6b8bb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:21.309588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95b480-c2ab-4a4f-b6e3-9699a6b8bb5a None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:21.309862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95b480-c2ab-4a4f-b6e3-9699a6b8bb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:21.310127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95b480-c2ab-4a4f-b6e3-9699a6b8bb5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:21.310817 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95b480-c2ab-4a4f-b6e3-9699a6b8bb5a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:21.311994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345003aa-51ca-46d3-bc6f-6736e3fd0ac6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:21.312167 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95b480-c2ab-4a4f-b6e3-9699a6b8bb5a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:21.330834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 389/781] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:30:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:21.331668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95b480-c2ab-4a4f-b6e3-9699a6b8bb5a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:21.364278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d572ef0-f0a5-4994-a813-6cc433fee344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:21.364909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d572ef0-f0a5-4994-a813-6cc433fee344 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:21.365194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d572ef0-f0a5-4994-a813-6cc433fee344 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:21.365466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d572ef0-f0a5-4994-a813-6cc433fee344 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:21.366109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d572ef0-f0a5-4994-a813-6cc433fee344 None admin] RBAC: Adding query filter params (type=image) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:21.366451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d572ef0-f0a5-4994-a813-6cc433fee344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:21.367872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d572ef0-f0a5-4994-a813-6cc433fee344 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:21.371422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:21.378350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 393/782] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:21.380047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d572ef0-f0a5-4994-a813-6cc433fee344 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:21.470305 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:21.474744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95b480-c2ab-4a4f-b6e3-9699a6b8bb5a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:21.503312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d572ef0-f0a5-4994-a813-6cc433fee344 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:21.515855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd63a79-6b64-41de-99ba-9b3c96ee406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:21.516480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd63a79-6b64-41de-99ba-9b3c96ee406b None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:21.516752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd63a79-6b64-41de-99ba-9b3c96ee406b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:21.517022 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd63a79-6b64-41de-99ba-9b3c96ee406b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:21.517756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd63a79-6b64-41de-99ba-9b3c96ee406b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:21.519192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd63a79-6b64-41de-99ba-9b3c96ee406b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:21.545101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 390/783] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:30:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:21.555717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a6a18-5afd-40c6-8faf-41894910f740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:21.556488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a6a18-5afd-40c6-8faf-41894910f740 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:21.556777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a6a18-5afd-40c6-8faf-41894910f740 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:21.557055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a6a18-5afd-40c6-8faf-41894910f740 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:21.557886 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a6a18-5afd-40c6-8faf-41894910f740 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:21.559334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a6a18-5afd-40c6-8faf-41894910f740 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:21.588078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 394/784] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:30:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:23.560525 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bdbbf-5228-4929-9b37-92c1d8fde896 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.561024 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bdbbf-5228-4929-9b37-92c1d8fde896 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.561102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bdbbf-5228-4929-9b37-92c1d8fde896 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.562592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 391/785] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:23.568642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c844a-563e-42f3-9541-d8cbcc61a0ae None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.569081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c844a-563e-42f3-9541-d8cbcc61a0ae None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.569223 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c844a-563e-42f3-9541-d8cbcc61a0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.588000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-628c844a-563e-42f3-9541-d8cbcc61a0ae None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:23.589163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-628c844a-563e-42f3-9541-d8cbcc61a0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:23.670584 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 395/786] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:23.789285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097585f-0aa6-48fe-93d2-e2c8088c48a6 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.789608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097585f-0aa6-48fe-93d2-e2c8088c48a6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.789866 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097585f-0aa6-48fe-93d2-e2c8088c48a6 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.791585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 392/787] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:23.796870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f969c2e-761a-4e67-bb3b-00e07418e943 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.797159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f969c2e-761a-4e67-bb3b-00e07418e943 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.797429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f969c2e-761a-4e67-bb3b-00e07418e943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.816330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3f969c2e-761a-4e67-bb3b-00e07418e943 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:23.817640 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3f969c2e-761a-4e67-bb3b-00e07418e943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:23.857420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1477e54-0c79-4b68-b976-2e415bd2d1eb None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.857758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1477e54-0c79-4b68-b976-2e415bd2d1eb None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.857989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1477e54-0c79-4b68-b976-2e415bd2d1eb None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.866585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 393/788] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:23.868355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41622f-d5b8-4f35-aa6c-a570da6e1aa2 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.868854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41622f-d5b8-4f35-aa6c-a570da6e1aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.869331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41622f-d5b8-4f35-aa6c-a570da6e1aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.886016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:23.904557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-db41622f-d5b8-4f35-aa6c-a570da6e1aa2 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:23.907523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-db41622f-d5b8-4f35-aa6c-a570da6e1aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:23.918584 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 396/789] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:23.951259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f969c2e-761a-4e67-bb3b-00e07418e943 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:23.986307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 394/790] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:24.094009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f969c2e-761a-4e67-bb3b-00e07418e943 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:24.167855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f903a-6f1b-4390-888f-cc7270499a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:24.168823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f903a-6f1b-4390-888f-cc7270499a52 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:24.169352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f903a-6f1b-4390-888f-cc7270499a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:24.169885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f903a-6f1b-4390-888f-cc7270499a52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:24.171230 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f903a-6f1b-4390-888f-cc7270499a52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:24.172684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f903a-6f1b-4390-888f-cc7270499a52 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:24.174311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 397/791] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:24.180865 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f903a-6f1b-4390-888f-cc7270499a52 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:24.189219 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41622f-d5b8-4f35-aa6c-a570da6e1aa2 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:24.212951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:24.241112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:24.302841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f903a-6f1b-4390-888f-cc7270499a52 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:24.315860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41622f-d5b8-4f35-aa6c-a570da6e1aa2 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:24.342778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabae63d-c781-4a3f-a6a5-63f66f5fecc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:24.343393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae63d-c781-4a3f-a6a5-63f66f5fecc4 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:24.343680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae63d-c781-4a3f-a6a5-63f66f5fecc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:24.343973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae63d-c781-4a3f-a6a5-63f66f5fecc4 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:24.344646 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabae63d-c781-4a3f-a6a5-63f66f5fecc4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:24.346014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabae63d-c781-4a3f-a6a5-63f66f5fecc4 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:24.360469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998491ec-70b3-4852-b066-a24ff13266d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:24.361732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998491ec-70b3-4852-b066-a24ff13266d0 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:24.362241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998491ec-70b3-4852-b066-a24ff13266d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:24.362525 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998491ec-70b3-4852-b066-a24ff13266d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:24.363218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998491ec-70b3-4852-b066-a24ff13266d0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:24.364693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998491ec-70b3-4852-b066-a24ff13266d0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:24.373133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.sql.core [None req-998491ec-70b3-4852-b066-a24ff13266d0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:30:24.373133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:30:24.373133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [parameters: {'id': '383078fa30d54630b1b51932bfc58e41', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 13:30:24.373133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81688) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:30:24.375122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 398/792] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:30:24 2022] POST /identity/v3/projects => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:30:24.375987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabae63d-c781-4a3f-a6a5-63f66f5fecc4 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-998491ec-70b3-4852-b066-a24ff13266d0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:30:24.382688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result.read() Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 13:30:24.383634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application yield resource Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise self.value Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application session.commit() Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:30:24.385038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:30:24.386068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:30:24.387224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application result.read() Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:30:24.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [parameters: {'id': '383078fa30d54630b1b51932bfc58e41', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 13:30:24.389445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:24.390400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:24.391735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 395/793] 10.222.0.20 () {60 vars in 1137 bytes} [Mon Oct 24 13:30:24 2022] POST /identity/v3/domains => generated 190 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:24.393654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998491ec-70b3-4852-b066-a24ff13266d0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:24.526065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:24.527037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:24.535222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabae63d-c781-4a3f-a6a5-63f66f5fecc4 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:24.559588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998491ec-70b3-4852-b066-a24ff13266d0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:24.586376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb0b4c-dec2-495d-ae10-186624ae47be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:24.587507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb0b4c-dec2-495d-ae10-186624ae47be None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:24.588498 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb0b4c-dec2-495d-ae10-186624ae47be None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:24.588498 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb0b4c-dec2-495d-ae10-186624ae47be None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:24.604547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb0b4c-dec2-495d-ae10-186624ae47be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-7bcb0b4c-dec2-495d-ae10-186624ae47be None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:30:24.609322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:24.610312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:24.611198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 399/794] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Oct 24 13:30:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:24.613992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b2d14-a410-4271-adb1-0d2234e0a873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:24.613992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b2d14-a410-4271-adb1-0d2234e0a873 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:24.614643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b2d14-a410-4271-adb1-0d2234e0a873 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:24.616304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b2d14-a410-4271-adb1-0d2234e0a873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:24.618712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb0b4c-dec2-495d-ae10-186624ae47be None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:24.632133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b2d14-a410-4271-adb1-0d2234e0a873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:24.634108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b2d14-a410-4271-adb1-0d2234e0a873 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-288b2d14-a410-4271-adb1-0d2234e0a873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:30:24.652617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:24.653666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:30:24.654649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:30:24.654649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:24.654649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:24.654649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 396/795] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:24.659920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b2d14-a410-4271-adb1-0d2234e0a873 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:24.754348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb0b4c-dec2-495d-ae10-186624ae47be None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:24.783767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b2d14-a410-4271-adb1-0d2234e0a873 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:24.807873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7002966-84a9-4b57-8427-49ddb284a32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:24.808493 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7002966-84a9-4b57-8427-49ddb284a32b None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:24.808770 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7002966-84a9-4b57-8427-49ddb284a32b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:24.809037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7002966-84a9-4b57-8427-49ddb284a32b None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:24.810249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7002966-84a9-4b57-8427-49ddb284a32b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:24.810884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7002966-84a9-4b57-8427-49ddb284a32b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:24.812524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7002966-84a9-4b57-8427-49ddb284a32b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:24.816236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:24.825410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 400/796] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Oct 24 13:30:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:24.831263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7002966-84a9-4b57-8427-49ddb284a32b None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:24.832908 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14cf4e-ec4e-4818-941f-d6860a240516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:24.833767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14cf4e-ec4e-4818-941f-d6860a240516 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:24.834057 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14cf4e-ec4e-4818-941f-d6860a240516 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:24.834425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14cf4e-ec4e-4818-941f-d6860a240516 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:24.835074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14cf4e-ec4e-4818-941f-d6860a240516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:24.835376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14cf4e-ec4e-4818-941f-d6860a240516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:24.836947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14cf4e-ec4e-4818-941f-d6860a240516 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:24.862329 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 397/797] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:24.878433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:25.010192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7002966-84a9-4b57-8427-49ddb284a32b None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:25.055299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0ca6d-1ae5-4602-9db8-a634353c18db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:25.055924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0ca6d-1ae5-4602-9db8-a634353c18db None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:25.056197 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0ca6d-1ae5-4602-9db8-a634353c18db None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:25.056458 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0ca6d-1ae5-4602-9db8-a634353c18db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:25.073405 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0ca6d-1ae5-4602-9db8-a634353c18db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:25.075158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0ca6d-1ae5-4602-9db8-a634353c18db None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-3eb0ca6d-1ae5-4602-9db8-a634353c18db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:30:25.091214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:25.092189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:30:25.093075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:30:25.093075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:25.093075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:25.093694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 401/798] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:25.103455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14cf4e-ec4e-4818-941f-d6860a240516 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:25.246660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14cf4e-ec4e-4818-941f-d6860a240516 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:25.280033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1967fe-f92e-40ba-aa7f-6f6dfd2fd16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:25.280647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1967fe-f92e-40ba-aa7f-6f6dfd2fd16b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:25.280924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1967fe-f92e-40ba-aa7f-6f6dfd2fd16b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:25.281192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1967fe-f92e-40ba-aa7f-6f6dfd2fd16b None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:25.281891 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1967fe-f92e-40ba-aa7f-6f6dfd2fd16b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:25.282149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1967fe-f92e-40ba-aa7f-6f6dfd2fd16b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:25.283532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1967fe-f92e-40ba-aa7f-6f6dfd2fd16b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:25.298226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 398/799] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:25.301851 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0ca6d-1ae5-4602-9db8-a634353c18db None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:25.384044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:25.410928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0ca6d-1ae5-4602-9db8-a634353c18db None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:25.444269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de512f-f5fd-4f0b-9d8f-982842d81faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:25.445095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de512f-f5fd-4f0b-9d8f-982842d81faa None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:25.445616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de512f-f5fd-4f0b-9d8f-982842d81faa None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:25.446088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de512f-f5fd-4f0b-9d8f-982842d81faa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:25.453459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de512f-f5fd-4f0b-9d8f-982842d81faa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:25.454989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de512f-f5fd-4f0b-9d8f-982842d81faa None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-89de512f-f5fd-4f0b-9d8f-982842d81faa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:30:25.465968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:25.467127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:25.468089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:25.468089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:25.468089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:30:25.468089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:30:25.468089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:30:25.468089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:30:25.468089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:30:25.468089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:30:25.468089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:30:25.468089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:25.468089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 402/800] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Oct 24 13:30:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:25.472812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1967fe-f92e-40ba-aa7f-6f6dfd2fd16b None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:25.522887 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:25.574302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1967fe-f92e-40ba-aa7f-6f6dfd2fd16b None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:25.607803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a90b4-677f-4207-84f2-289ffe90c4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:25.608410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a90b4-677f-4207-84f2-289ffe90c4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:25.608686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a90b4-677f-4207-84f2-289ffe90c4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:25.608961 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a90b4-677f-4207-84f2-289ffe90c4c2 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:25.609660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a90b4-677f-4207-84f2-289ffe90c4c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:25.609958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a90b4-677f-4207-84f2-289ffe90c4c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:25.611418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a90b4-677f-4207-84f2-289ffe90c4c2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:25.628911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 399/801] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Oct 24 13:30:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:25.635478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de512f-f5fd-4f0b-9d8f-982842d81faa None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:25.737961 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de512f-f5fd-4f0b-9d8f-982842d81faa None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:25.770564 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1a54d-2af7-4d8c-a60f-9195d6fccffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:25.771184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1a54d-2af7-4d8c-a60f-9195d6fccffd None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:25.771458 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1a54d-2af7-4d8c-a60f-9195d6fccffd None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:25.771723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1a54d-2af7-4d8c-a60f-9195d6fccffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:25.773532 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:25.782554 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1a54d-2af7-4d8c-a60f-9195d6fccffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:25.783969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1a54d-2af7-4d8c-a60f-9195d6fccffd None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-fbc1a54d-2af7-4d8c-a60f-9195d6fccffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:30:25.798262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:25.799186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:30:25.800105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:30:25.800105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:25.800105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:25.800105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 403/802] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:25.805016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a90b4-677f-4207-84f2-289ffe90c4c2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:25.903909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a90b4-677f-4207-84f2-289ffe90c4c2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:25.911019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:25.937942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238c1c1-d533-402e-9097-72238f24ca67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:25.938582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238c1c1-d533-402e-9097-72238f24ca67 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:25.938862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238c1c1-d533-402e-9097-72238f24ca67 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:25.939129 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238c1c1-d533-402e-9097-72238f24ca67 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:25.939765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238c1c1-d533-402e-9097-72238f24ca67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:25.940066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238c1c1-d533-402e-9097-72238f24ca67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:25.942000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238c1c1-d533-402e-9097-72238f24ca67 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:25.953181 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 400/803] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:25.959524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1a54d-2af7-4d8c-a60f-9195d6fccffd None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:26.057984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1a54d-2af7-4d8c-a60f-9195d6fccffd None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:26.091866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b222b77-f2c5-4fd0-aced-1e02925abf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:26.092488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b222b77-f2c5-4fd0-aced-1e02925abf07 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:26.092767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b222b77-f2c5-4fd0-aced-1e02925abf07 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:26.093037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b222b77-f2c5-4fd0-aced-1e02925abf07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:26.099706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b222b77-f2c5-4fd0-aced-1e02925abf07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:26.101125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b222b77-f2c5-4fd0-aced-1e02925abf07 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-5b222b77-f2c5-4fd0-aced-1e02925abf07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:30:26.112166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:26.113207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:26.114151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:30:26.114151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:30:26.114151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:30:26.114151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:30:26.114151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:30:26.114151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:30:26.114151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:30:26.114151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:26.114151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 404/804] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:30:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:26.119126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238c1c1-d533-402e-9097-72238f24ca67 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:26.220736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238c1c1-d533-402e-9097-72238f24ca67 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:26.256489 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d98229-723f-48c0-9b4f-a0b5d3e88136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:26.257096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d98229-723f-48c0-9b4f-a0b5d3e88136 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:26.257372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d98229-723f-48c0-9b4f-a0b5d3e88136 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:26.257642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d98229-723f-48c0-9b4f-a0b5d3e88136 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:26.258356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d98229-723f-48c0-9b4f-a0b5d3e88136 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:26.258656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d98229-723f-48c0-9b4f-a0b5d3e88136 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:26.260086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d98229-723f-48c0-9b4f-a0b5d3e88136 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:26.272513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 401/805] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:30:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:26.278422 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b222b77-f2c5-4fd0-aced-1e02925abf07 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:26.324402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:26.380153 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b222b77-f2c5-4fd0-aced-1e02925abf07 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:26.413107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0390a5-870b-4e5b-bb0e-a8933d8cf5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:26.413759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0390a5-870b-4e5b-bb0e-a8933d8cf5a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:26.414037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0390a5-870b-4e5b-bb0e-a8933d8cf5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:26.414335 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0390a5-870b-4e5b-bb0e-a8933d8cf5a3 None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/18bb93669846435a85eb34771cfb09fe/roles/79ebc57a5a9041d08f54488fb7845f5e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:26.416698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0390a5-870b-4e5b-bb0e-a8933d8cf5a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=18bb93669846435a85eb34771cfb09fe, role_id=79ebc57a5a9041d08f54488fb7845f5e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:26.418132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0390a5-870b-4e5b-bb0e-a8933d8cf5a3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:26.430946 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 405/806] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:30:26 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/18bb93669846435a85eb34771cfb09fe/roles/79ebc57a5a9041d08f54488fb7845f5e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:30:26.639722 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a5538-2b4f-464c-9072-dfaf01433b6d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:26.640227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a5538-2b4f-464c-9072-dfaf01433b6d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:26.640227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a5538-2b4f-464c-9072-dfaf01433b6d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:26.641523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 402/807] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:26.647547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374ed0-10de-4b81-96e3-078fbfc558b9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:26.647973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374ed0-10de-4b81-96e3-078fbfc558b9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:26.648215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374ed0-10de-4b81-96e3-078fbfc558b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:26.668978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b4374ed0-10de-4b81-96e3-078fbfc558b9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:26.670150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b4374ed0-10de-4b81-96e3-078fbfc558b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:26.737068 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 406/808] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:27.006141 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a5538-2b4f-464c-9072-dfaf01433b6d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:27.033262 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:27.107422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a5538-2b4f-464c-9072-dfaf01433b6d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:27.141741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750017f-0d3b-422f-ba19-e819bf04c31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:27.142399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750017f-0d3b-422f-ba19-e819bf04c31d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:27.142679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750017f-0d3b-422f-ba19-e819bf04c31d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:27.142950 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750017f-0d3b-422f-ba19-e819bf04c31d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:27.144052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750017f-0d3b-422f-ba19-e819bf04c31d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:27.145524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750017f-0d3b-422f-ba19-e819bf04c31d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:27.147125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 403/809] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:27.152873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4374ed0-10de-4b81-96e3-078fbfc558b9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:27.187475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:27.257427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4374ed0-10de-4b81-96e3-078fbfc558b9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:27.290578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ade9a-e9d5-49da-9612-f00691259eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:27.291189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ade9a-e9d5-49da-9612-f00691259eea None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:27.291470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ade9a-e9d5-49da-9612-f00691259eea None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:27.291741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ade9a-e9d5-49da-9612-f00691259eea None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:27.292398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ade9a-e9d5-49da-9612-f00691259eea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:27.293815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ade9a-e9d5-49da-9612-f00691259eea None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:27.320314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 407/810] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:30:27 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:28.577679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822e3ad-10cf-4fc8-b285-60ddcb73611a None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:28.578218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822e3ad-10cf-4fc8-b285-60ddcb73611a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:28.578325 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822e3ad-10cf-4fc8-b285-60ddcb73611a None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:28.579596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 404/811] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:28.585188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed74ba1-e6ac-4c68-a5cd-620ffd53cb9b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:28.585635 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed74ba1-e6ac-4c68-a5cd-620ffd53cb9b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:28.585725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed74ba1-e6ac-4c68-a5cd-620ffd53cb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:28.603916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-aed74ba1-e6ac-4c68-a5cd-620ffd53cb9b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:28.605102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-aed74ba1-e6ac-4c68-a5cd-620ffd53cb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:28.671875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 408/812] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:28.944513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822e3ad-10cf-4fc8-b285-60ddcb73611a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:29.038805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:29.047155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822e3ad-10cf-4fc8-b285-60ddcb73611a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:29.080475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b82e30-d223-4b05-9cfa-4af1c6d1c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:29.081092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b82e30-d223-4b05-9cfa-4af1c6d1c66a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:29.081365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b82e30-d223-4b05-9cfa-4af1c6d1c66a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:29.081630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b82e30-d223-4b05-9cfa-4af1c6d1c66a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:29.091467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b82e30-d223-4b05-9cfa-4af1c6d1c66a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-f7b82e30-d223-4b05-9cfa-4af1c6d1c66a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:30:29.096179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:29.097267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:29.099227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 405/813] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Oct 24 13:30:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:29.103755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed74ba1-e6ac-4c68-a5cd-620ffd53cb9b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:29.130648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:29.207528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed74ba1-e6ac-4c68-a5cd-620ffd53cb9b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:29.244030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876e244-a590-4289-a354-3f279a4794d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:29.244633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876e244-a590-4289-a354-3f279a4794d0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:29.244920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876e244-a590-4289-a354-3f279a4794d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:29.245207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876e244-a590-4289-a354-3f279a4794d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:29.245846 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876e244-a590-4289-a354-3f279a4794d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:29.246147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876e244-a590-4289-a354-3f279a4794d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:29.247530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876e244-a590-4289-a354-3f279a4794d0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:29.255493 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 409/814] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Oct 24 13:30:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:29.261296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b82e30-d223-4b05-9cfa-4af1c6d1c66a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:29.360140 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b82e30-d223-4b05-9cfa-4af1c6d1c66a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:29.397157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06459f8-dbac-4461-948e-1afc187ca286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:29.397807 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06459f8-dbac-4461-948e-1afc187ca286 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:29.398087 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06459f8-dbac-4461-948e-1afc187ca286 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:29.398385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06459f8-dbac-4461-948e-1afc187ca286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:29.409410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06459f8-dbac-4461-948e-1afc187ca286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:29.411439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06459f8-dbac-4461-948e-1afc187ca286 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-b06459f8-dbac-4461-948e-1afc187ca286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:30:29.426061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:29.427081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:30:29.427997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:30:29.427997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:29.427997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:29.427997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 406/815] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:29.433143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876e244-a590-4289-a354-3f279a4794d0 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:29.492432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff253e-b841-4f41-8302-fe407f064e1e None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:29.492778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff253e-b841-4f41-8302-fe407f064e1e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:29.493117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff253e-b841-4f41-8302-fe407f064e1e None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:29.494602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 407/816] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:30:29.499337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6da1e1-5eb8-421e-a652-e8874709aeff None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:29.499614 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6da1e1-5eb8-421e-a652-e8874709aeff None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:29.499882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6da1e1-5eb8-421e-a652-e8874709aeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:29.512353 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:29.517947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6f6da1e1-5eb8-421e-a652-e8874709aeff None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:29.518377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:29.519240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6f6da1e1-5eb8-421e-a652-e8874709aeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:29.535197 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876e244-a590-4289-a354-3f279a4794d0 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:29.571380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01054ea0-236a-4065-bbd6-614437520099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:29.571987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01054ea0-236a-4065-bbd6-614437520099 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:29.572261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01054ea0-236a-4065-bbd6-614437520099 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:29.572524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01054ea0-236a-4065-bbd6-614437520099 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:29.573154 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01054ea0-236a-4065-bbd6-614437520099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:29.573451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01054ea0-236a-4065-bbd6-614437520099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:29.574822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01054ea0-236a-4065-bbd6-614437520099 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:29.586629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 410/817] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:29.590755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 408/818] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:29.593147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01054ea0-236a-4065-bbd6-614437520099 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:29.692057 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01054ea0-236a-4065-bbd6-614437520099 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:29.725129 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edef67-d5f4-4718-9964-675fc7561b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:29.725743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edef67-d5f4-4718-9964-675fc7561b89 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:29.726016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edef67-d5f4-4718-9964-675fc7561b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:29.726324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edef67-d5f4-4718-9964-675fc7561b89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:29.732939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edef67-d5f4-4718-9964-675fc7561b89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:29.734391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edef67-d5f4-4718-9964-675fc7561b89 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-79edef67-d5f4-4718-9964-675fc7561b89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:30:29.745538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:29.746639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:29.747566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:29.747566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:29.747566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:30:29.747566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:30:29.747566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:30:29.747566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:30:29.747566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:30:29.747566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:30:29.747566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:30:29.747566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:29.752340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 411/819] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Oct 24 13:30:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:29.753068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6da1e1-5eb8-421e-a652-e8874709aeff None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:29.853188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6da1e1-5eb8-421e-a652-e8874709aeff None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:29.856168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edef67-d5f4-4718-9964-675fc7561b89 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:29.860290 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:29.887846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e942959-78cf-491a-bcdf-d2ee64d46aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:29.888459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e942959-78cf-491a-bcdf-d2ee64d46aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:29.888759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e942959-78cf-491a-bcdf-d2ee64d46aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:29.889024 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e942959-78cf-491a-bcdf-d2ee64d46aa5 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:29.889741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e942959-78cf-491a-bcdf-d2ee64d46aa5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:29.890037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e942959-78cf-491a-bcdf-d2ee64d46aa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:29.891495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e942959-78cf-491a-bcdf-d2ee64d46aa5 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:29.893962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:29.909585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 409/820] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Oct 24 13:30:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:29.915718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e942959-78cf-491a-bcdf-d2ee64d46aa5 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:29.961234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edef67-d5f4-4718-9964-675fc7561b89 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:29.996138 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57ab70-9310-4fec-aa21-4017c8825424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:29.996690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57ab70-9310-4fec-aa21-4017c8825424 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:29.996969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57ab70-9310-4fec-aa21-4017c8825424 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:29.997241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57ab70-9310-4fec-aa21-4017c8825424 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:30.008972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57ab70-9310-4fec-aa21-4017c8825424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:30:30.012623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e942959-78cf-491a-bcdf-d2ee64d46aa5 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-cb57ab70-9310-4fec-aa21-4017c8825424 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:30:30.013764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:30.014775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:30.015781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 412/821] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:30:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:30.020908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57ab70-9310-4fec-aa21-4017c8825424 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:30.047457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503cad3-9fb2-4b01-b3b8-2e2a7e041883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:30.048063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503cad3-9fb2-4b01-b3b8-2e2a7e041883 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:30.048340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503cad3-9fb2-4b01-b3b8-2e2a7e041883 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:30.048610 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503cad3-9fb2-4b01-b3b8-2e2a7e041883 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:30.084511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:30.109780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:30.118186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503cad3-9fb2-4b01-b3b8-2e2a7e041883 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:30.119646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503cad3-9fb2-4b01-b3b8-2e2a7e041883 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:30.123371 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57ab70-9310-4fec-aa21-4017c8825424 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:30.157269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40746a6d-2ce7-4c31-b774-8ed4ffef8979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:30.157877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746a6d-2ce7-4c31-b774-8ed4ffef8979 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:30.158201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746a6d-2ce7-4c31-b774-8ed4ffef8979 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:30.158478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40746a6d-2ce7-4c31-b774-8ed4ffef8979 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:30.159135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40746a6d-2ce7-4c31-b774-8ed4ffef8979 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:30.159441 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40746a6d-2ce7-4c31-b774-8ed4ffef8979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:30.160819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40746a6d-2ce7-4c31-b774-8ed4ffef8979 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:30.169545 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 413/822] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:30:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:30.175276 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40746a6d-2ce7-4c31-b774-8ed4ffef8979 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:30.220956 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 410/823] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:30:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 24 13:30:30.226565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503cad3-9fb2-4b01-b3b8-2e2a7e041883 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:30.283881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40746a6d-2ce7-4c31-b774-8ed4ffef8979 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:30.310393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:30.319977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:30.320115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78db2e-bf46-4540-a304-32b69ae17094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:30.321146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78db2e-bf46-4540-a304-32b69ae17094 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:30.321457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78db2e-bf46-4540-a304-32b69ae17094 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:30.321870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78db2e-bf46-4540-a304-32b69ae17094 None admin] PATH_INFO: `/v3/users/90f4100545aa4c6eb1af80853febedc6` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:30.326940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503cad3-9fb2-4b01-b3b8-2e2a7e041883 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:30.335037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78db2e-bf46-4540-a304-32b69ae17094 None admin] RBAC: Authorizing `identity:get_user(user_id=90f4100545aa4c6eb1af80853febedc6)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:30.336469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78db2e-bf46-4540-a304-32b69ae17094 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:30.338132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 414/824] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:30:30 2022] GET /identity/v3/users/90f4100545aa4c6eb1af80853febedc6 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:30.343790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78db2e-bf46-4540-a304-32b69ae17094 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:30.361152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24afa6b-fe4f-4ca3-bf66-2b0033e6a150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:30.361774 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24afa6b-fe4f-4ca3-bf66-2b0033e6a150 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:30.362044 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24afa6b-fe4f-4ca3-bf66-2b0033e6a150 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:30.362347 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24afa6b-fe4f-4ca3-bf66-2b0033e6a150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:30.372846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24afa6b-fe4f-4ca3-bf66-2b0033e6a150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:30.374517 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24afa6b-fe4f-4ca3-bf66-2b0033e6a150 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-c24afa6b-fe4f-4ca3-bf66-2b0033e6a150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:30:30.388679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:30.389823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:30:30.390904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:30:30.390904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:30:30.390904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:30.390904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 411/825] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:30.395546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24afa6b-fe4f-4ca3-bf66-2b0033e6a150 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:30.447094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78db2e-bf46-4540-a304-32b69ae17094 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:30.481229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf12ee-54eb-4b7f-8224-96a5ca9734fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:30.481937 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf12ee-54eb-4b7f-8224-96a5ca9734fc None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:30.482328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf12ee-54eb-4b7f-8224-96a5ca9734fc None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:30.482660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf12ee-54eb-4b7f-8224-96a5ca9734fc None admin] PATH_INFO: `/v3/projects/fc4f03999e85403bb0fc55c89f6709a9` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:30.483811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf12ee-54eb-4b7f-8224-96a5ca9734fc None admin] RBAC: Authorizing `identity:get_project(project_id=fc4f03999e85403bb0fc55c89f6709a9)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:30.485216 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf12ee-54eb-4b7f-8224-96a5ca9734fc None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:30.486839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 415/826] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:30:30 2022] GET /identity/v3/projects/fc4f03999e85403bb0fc55c89f6709a9 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:30.492885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf12ee-54eb-4b7f-8224-96a5ca9734fc None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:30.494653 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24afa6b-fe4f-4ca3-bf66-2b0033e6a150 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:30.530933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c2ea3-3255-4452-8053-b1af9bc11670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:30.531551 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c2ea3-3255-4452-8053-b1af9bc11670 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:30.531833 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c2ea3-3255-4452-8053-b1af9bc11670 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:30.532105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c2ea3-3255-4452-8053-b1af9bc11670 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:30.532741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c2ea3-3255-4452-8053-b1af9bc11670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:30.533043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c2ea3-3255-4452-8053-b1af9bc11670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:30.534499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c2ea3-3255-4452-8053-b1af9bc11670 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:30.544253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:30.546902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 412/827] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:30.552667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c2ea3-3255-4452-8053-b1af9bc11670 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:30.560728 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:30.592900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf12ee-54eb-4b7f-8224-96a5ca9734fc None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:30.625730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa1887-bec6-44c1-b38c-7607a39e98c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:30.626411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa1887-bec6-44c1-b38c-7607a39e98c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:30.626683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa1887-bec6-44c1-b38c-7607a39e98c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:30.626949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa1887-bec6-44c1-b38c-7607a39e98c7 None admin] PATH_INFO: `/v3/projects/fc4f03999e85403bb0fc55c89f6709a9/users/90f4100545aa4c6eb1af80853febedc6/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:30.629390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa1887-bec6-44c1-b38c-7607a39e98c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc4f03999e85403bb0fc55c89f6709a9, user_id=90f4100545aa4c6eb1af80853febedc6, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:30.630821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa1887-bec6-44c1-b38c-7607a39e98c7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:30.643130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 416/828] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:30:30 2022] PUT /identity/v3/projects/fc4f03999e85403bb0fc55c89f6709a9/users/90f4100545aa4c6eb1af80853febedc6/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:30:30.652972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c2ea3-3255-4452-8053-b1af9bc11670 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:30.689477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce103c-8b01-4baa-b6a0-9b538cd463c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:30.690112 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce103c-8b01-4baa-b6a0-9b538cd463c2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:30.690432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce103c-8b01-4baa-b6a0-9b538cd463c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:30.690700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce103c-8b01-4baa-b6a0-9b538cd463c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:30.698770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce103c-8b01-4baa-b6a0-9b538cd463c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:30.700244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce103c-8b01-4baa-b6a0-9b538cd463c2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-94ce103c-8b01-4baa-b6a0-9b538cd463c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:30:30.713389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:30.714541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:30.715415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:30:30.715415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:30:30.715415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:30:30.715415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:30:30.715415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:30:30.715415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:30:30.715415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:30:30.715415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:30.715415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 413/829] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:30:30.720638 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa1887-bec6-44c1-b38c-7607a39e98c7 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:30.830360 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa1887-bec6-44c1-b38c-7607a39e98c7 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:30.844518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:30.868524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bf7ff-ba7f-43b9-b099-69fd4b7baabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:30.869129 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bf7ff-ba7f-43b9-b099-69fd4b7baabf None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:30.869403 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bf7ff-ba7f-43b9-b099-69fd4b7baabf None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:30.869669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bf7ff-ba7f-43b9-b099-69fd4b7baabf None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:30.870376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2bf7ff-ba7f-43b9-b099-69fd4b7baabf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:30.870675 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2bf7ff-ba7f-43b9-b099-69fd4b7baabf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:30.872088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2bf7ff-ba7f-43b9-b099-69fd4b7baabf None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:30.884382 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 417/830] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:30:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:30.890735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce103c-8b01-4baa-b6a0-9b538cd463c2 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:30.959406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:30.996087 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce103c-8b01-4baa-b6a0-9b538cd463c2 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:31.029360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef10371-476e-4a61-9226-a8ebae57e88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:31.029964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef10371-476e-4a61-9226-a8ebae57e88b None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:31.030272 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef10371-476e-4a61-9226-a8ebae57e88b None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:31.030541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef10371-476e-4a61-9226-a8ebae57e88b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:31.031260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef10371-476e-4a61-9226-a8ebae57e88b None admin] RBAC: Adding query filter params (role.id=79ebc57a5a9041d08f54488fb7845f5e, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=18bb93669846435a85eb34771cfb09fe) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:31.031605 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef10371-476e-4a61-9226-a8ebae57e88b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:31.033011 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef10371-476e-4a61-9226-a8ebae57e88b None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:31.039515 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 414/831] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:30:30 2022] GET /identity/v3/role_assignments?user.id=18bb93669846435a85eb34771cfb09fe&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=79ebc57a5a9041d08f54488fb7845f5e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:32.807864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba054b-9cad-4a79-8193-ae06d7d8a96e None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:32.808387 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba054b-9cad-4a79-8193-ae06d7d8a96e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:32.808387 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba054b-9cad-4a79-8193-ae06d7d8a96e None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:32.809746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 418/832] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:32.814981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faab9c9-718f-4dec-84b8-4ae1b33d9ea4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:32.815401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faab9c9-718f-4dec-84b8-4ae1b33d9ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:32.815530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faab9c9-718f-4dec-84b8-4ae1b33d9ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:32.835081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9faab9c9-718f-4dec-84b8-4ae1b33d9ea4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:32.836269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9faab9c9-718f-4dec-84b8-4ae1b33d9ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:32.900340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 415/833] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:33.171755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba054b-9cad-4a79-8193-ae06d7d8a96e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:33.271444 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba054b-9cad-4a79-8193-ae06d7d8a96e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:33.304372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33482f-7ba4-45d6-8251-bcbfa3a63210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:33.304992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33482f-7ba4-45d6-8251-bcbfa3a63210 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:33.305273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33482f-7ba4-45d6-8251-bcbfa3a63210 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:33.305541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33482f-7ba4-45d6-8251-bcbfa3a63210 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:33.315674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33482f-7ba4-45d6-8251-bcbfa3a63210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-9c33482f-7ba4-45d6-8251-bcbfa3a63210 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:30:33.321061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:33.322131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:33.323082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 419/834] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:30:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:33.328284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faab9c9-718f-4dec-84b8-4ae1b33d9ea4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:33.384807 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:33.424337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faab9c9-718f-4dec-84b8-4ae1b33d9ea4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:33.457558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd5c14-4db4-44ec-9090-c42a80fd13da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:33.458142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd5c14-4db4-44ec-9090-c42a80fd13da None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:33.458446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd5c14-4db4-44ec-9090-c42a80fd13da None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:33.458732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd5c14-4db4-44ec-9090-c42a80fd13da None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:33.459363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd5c14-4db4-44ec-9090-c42a80fd13da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:33.459655 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd5c14-4db4-44ec-9090-c42a80fd13da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:33.460996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd5c14-4db4-44ec-9090-c42a80fd13da None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:33.469254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 416/835] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:30:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:33.475425 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33482f-7ba4-45d6-8251-bcbfa3a63210 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:33.506245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:33.572854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33482f-7ba4-45d6-8251-bcbfa3a63210 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:33.605465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800b2f7-a376-4dbe-bfe5-2b46bbd06ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:33.606080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800b2f7-a376-4dbe-bfe5-2b46bbd06ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:33.606389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800b2f7-a376-4dbe-bfe5-2b46bbd06ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:33.606657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800b2f7-a376-4dbe-bfe5-2b46bbd06ab7 None admin] PATH_INFO: `/v3/users/90f4100545aa4c6eb1af80853febedc6` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:33.607738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800b2f7-a376-4dbe-bfe5-2b46bbd06ab7 None admin] RBAC: Authorizing `identity:get_user(user_id=90f4100545aa4c6eb1af80853febedc6)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:33.609149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800b2f7-a376-4dbe-bfe5-2b46bbd06ab7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:33.610707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 420/836] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:30:33 2022] GET /identity/v3/users/90f4100545aa4c6eb1af80853febedc6 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:33.617515 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd5c14-4db4-44ec-9090-c42a80fd13da None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:33.717839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd5c14-4db4-44ec-9090-c42a80fd13da None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:33.739120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:33.750258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f398b-e4bc-4cd3-94b4-7da38ac2003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:33.750852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f398b-e4bc-4cd3-94b4-7da38ac2003e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:33.751126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f398b-e4bc-4cd3-94b4-7da38ac2003e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:33.751388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f398b-e4bc-4cd3-94b4-7da38ac2003e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:33.815830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f398b-e4bc-4cd3-94b4-7da38ac2003e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:33.817960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f398b-e4bc-4cd3-94b4-7da38ac2003e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:33.911682 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 417/837] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:30:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:30:33.917906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800b2f7-a376-4dbe-bfe5-2b46bbd06ab7 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:34.014811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:34.017747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800b2f7-a376-4dbe-bfe5-2b46bbd06ab7 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:34.051526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab573595-0ca6-4d09-aff0-e2495be657d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:34.052139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab573595-0ca6-4d09-aff0-e2495be657d3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:34.052417 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab573595-0ca6-4d09-aff0-e2495be657d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:34.052687 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab573595-0ca6-4d09-aff0-e2495be657d3 None admin] PATH_INFO: `/v3/projects/fc4f03999e85403bb0fc55c89f6709a9` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:34.053786 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab573595-0ca6-4d09-aff0-e2495be657d3 None admin] RBAC: Authorizing `identity:get_project(project_id=fc4f03999e85403bb0fc55c89f6709a9)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:34.055227 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab573595-0ca6-4d09-aff0-e2495be657d3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:34.056826 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 421/838] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:30:33 2022] GET /identity/v3/projects/fc4f03999e85403bb0fc55c89f6709a9 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:34.062961 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f398b-e4bc-4cd3-94b4-7da38ac2003e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:34.086270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:34.159469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f398b-e4bc-4cd3-94b4-7da38ac2003e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:34.192853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb6545-ab64-4728-8b6d-a6ca71da0869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:34.193445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb6545-ab64-4728-8b6d-a6ca71da0869 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:34.193720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb6545-ab64-4728-8b6d-a6ca71da0869 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:34.193987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb6545-ab64-4728-8b6d-a6ca71da0869 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:34.194729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb6545-ab64-4728-8b6d-a6ca71da0869 None admin] RBAC: Adding query filter params (role.id=8b32d66a3cb3483c8ede361ad8637f66, scope.project.id=fc4f03999e85403bb0fc55c89f6709a9, user.id=90f4100545aa4c6eb1af80853febedc6) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:34.195029 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb6545-ab64-4728-8b6d-a6ca71da0869 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:34.196448 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb6545-ab64-4728-8b6d-a6ca71da0869 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:34.202126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 418/839] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:30:34 2022] GET /identity/v3/role_assignments?user.id=90f4100545aa4c6eb1af80853febedc6&scope.project.id=fc4f03999e85403bb0fc55c89f6709a9&role.id=8b32d66a3cb3483c8ede361ad8637f66 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:36.346709 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b8c49-fab4-4d25-93a3-8cf0468737b3 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:36.347214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b8c49-fab4-4d25-93a3-8cf0468737b3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:36.347297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b8c49-fab4-4d25-93a3-8cf0468737b3 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:36.348834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 422/840] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:36.354180 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb083b-4ebe-4530-8798-f3fa96a06233 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:36.354619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb083b-4ebe-4530-8798-f3fa96a06233 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:36.354714 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb083b-4ebe-4530-8798-f3fa96a06233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:36.372281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8bcb083b-4ebe-4530-8798-f3fa96a06233 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:36.373442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8bcb083b-4ebe-4530-8798-f3fa96a06233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:36.437538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 419/841] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:30:36.708394 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b8c49-fab4-4d25-93a3-8cf0468737b3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:36.924800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b8c49-fab4-4d25-93a3-8cf0468737b3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:36.958800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bd01c-cdec-4173-911b-2fc3d5e75c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:36.959365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bd01c-cdec-4173-911b-2fc3d5e75c77 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:36.959643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bd01c-cdec-4173-911b-2fc3d5e75c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:36.959911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bd01c-cdec-4173-911b-2fc3d5e75c77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:36.961050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bd01c-cdec-4173-911b-2fc3d5e75c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:36.962481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bd01c-cdec-4173-911b-2fc3d5e75c77 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:36.964030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 423/842] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:36.969854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb083b-4ebe-4530-8798-f3fa96a06233 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:36.987659 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:37.065561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb083b-4ebe-4530-8798-f3fa96a06233 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:37.099139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab248ef-8436-4fad-b9b1-3cdb972dd7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:37.099734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab248ef-8436-4fad-b9b1-3cdb972dd7db None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:37.100007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab248ef-8436-4fad-b9b1-3cdb972dd7db None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:37.100270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab248ef-8436-4fad-b9b1-3cdb972dd7db None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:37.100919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab248ef-8436-4fad-b9b1-3cdb972dd7db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:37.102348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab248ef-8436-4fad-b9b1-3cdb972dd7db None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:37.123168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 420/843] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:30:36 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:30:39.297378 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be47386-fc92-436c-9cfe-88a96190ae36 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:39.297871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be47386-fc92-436c-9cfe-88a96190ae36 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:39.297957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be47386-fc92-436c-9cfe-88a96190ae36 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:39.299297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 424/844] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:39.304717 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a669c-6a28-4413-b186-6a1d030a7368 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:39.305124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a669c-6a28-4413-b186-6a1d030a7368 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:39.305267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a669c-6a28-4413-b186-6a1d030a7368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:39.323682 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2a7a669c-6a28-4413-b186-6a1d030a7368 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:39.324855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2a7a669c-6a28-4413-b186-6a1d030a7368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:39.389344 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 421/845] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:39.661055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be47386-fc92-436c-9cfe-88a96190ae36 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:39.697545 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:39.762547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be47386-fc92-436c-9cfe-88a96190ae36 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:39.795021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1caed-b794-41ef-9eb9-f08d5656e38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:39.795636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1caed-b794-41ef-9eb9-f08d5656e38f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:39.795910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1caed-b794-41ef-9eb9-f08d5656e38f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:39.796173 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1caed-b794-41ef-9eb9-f08d5656e38f None admin] PATH_INFO: `/v3/roles/f948e6f6ee4f42de92195aee35b05641` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:39.797521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1caed-b794-41ef-9eb9-f08d5656e38f None admin] RBAC: Authorizing `identity:get_role(role_id=f948e6f6ee4f42de92195aee35b05641)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:39.798883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1caed-b794-41ef-9eb9-f08d5656e38f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:39.800171 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 425/846] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:30:39 2022] GET /identity/v3/roles/f948e6f6ee4f42de92195aee35b05641 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:39.806718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a669c-6a28-4413-b186-6a1d030a7368 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:39.808865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:39.905451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a669c-6a28-4413-b186-6a1d030a7368 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:40.055167 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e708f-4ad5-4cc5-8e77-d3872a4e11b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:40.055809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e708f-4ad5-4cc5-8e77-d3872a4e11b9 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:40.056090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e708f-4ad5-4cc5-8e77-d3872a4e11b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:40.056362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e708f-4ad5-4cc5-8e77-d3872a4e11b9 None admin] PATH_INFO: `/v3/users/889a2b61eb854d58a09982c1e6c5c0dc` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:40.070182 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e708f-4ad5-4cc5-8e77-d3872a4e11b9 None admin] RBAC: Authorizing `identity:get_user(user_id=889a2b61eb854d58a09982c1e6c5c0dc)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:40.071625 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e708f-4ad5-4cc5-8e77-d3872a4e11b9 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:40.073244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 422/847] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:30:39 2022] GET /identity/v3/users/889a2b61eb854d58a09982c1e6c5c0dc => generated 286 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:40.079144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1caed-b794-41ef-9eb9-f08d5656e38f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:40.186492 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1caed-b794-41ef-9eb9-f08d5656e38f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:40.189491 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:40.222868 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bcd31-ded6-4cb8-bed4-8f60d2296874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:40.223512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bcd31-ded6-4cb8-bed4-8f60d2296874 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:40.223788 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bcd31-ded6-4cb8-bed4-8f60d2296874 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:40.224053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bcd31-ded6-4cb8-bed4-8f60d2296874 None admin] PATH_INFO: `/v3/projects/fc4f03999e85403bb0fc55c89f6709a9` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:40.225156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04bcd31-ded6-4cb8-bed4-8f60d2296874 None admin] RBAC: Authorizing `identity:get_project(project_id=fc4f03999e85403bb0fc55c89f6709a9)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:40.226616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04bcd31-ded6-4cb8-bed4-8f60d2296874 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:40.228242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 426/848] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:30:40 2022] GET /identity/v3/projects/fc4f03999e85403bb0fc55c89f6709a9 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:40.234399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e708f-4ad5-4cc5-8e77-d3872a4e11b9 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:40.295285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:40.336227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e708f-4ad5-4cc5-8e77-d3872a4e11b9 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:40.370435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf98477-7355-4276-932d-9406794646b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:40.371090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf98477-7355-4276-932d-9406794646b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:40.371370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf98477-7355-4276-932d-9406794646b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:40.371642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf98477-7355-4276-932d-9406794646b3 None admin] PATH_INFO: `/v3/projects/fc4f03999e85403bb0fc55c89f6709a9/users/889a2b61eb854d58a09982c1e6c5c0dc/roles/f948e6f6ee4f42de92195aee35b05641` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:40.373851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf98477-7355-4276-932d-9406794646b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc4f03999e85403bb0fc55c89f6709a9, user_id=889a2b61eb854d58a09982c1e6c5c0dc, role_id=f948e6f6ee4f42de92195aee35b05641)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:40.375304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf98477-7355-4276-932d-9406794646b3 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:40.389459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 423/849] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:30:40 2022] PUT /identity/v3/projects/fc4f03999e85403bb0fc55c89f6709a9/users/889a2b61eb854d58a09982c1e6c5c0dc/roles/f948e6f6ee4f42de92195aee35b05641 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:30:42.544177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7417cff-21f0-4f81-8027-57a557a2bf21 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:42.544690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7417cff-21f0-4f81-8027-57a557a2bf21 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:42.544690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7417cff-21f0-4f81-8027-57a557a2bf21 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:42.546005 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 427/850] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:42.551823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17a2e2-6713-47cc-b8e5-72a483c0582a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:42.552247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17a2e2-6713-47cc-b8e5-72a483c0582a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:42.552338 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17a2e2-6713-47cc-b8e5-72a483c0582a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:42.571072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-be17a2e2-6713-47cc-b8e5-72a483c0582a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:42.572237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-be17a2e2-6713-47cc-b8e5-72a483c0582a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:42.639586 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 424/851] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:42.908582 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7417cff-21f0-4f81-8027-57a557a2bf21 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:43.012482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7417cff-21f0-4f81-8027-57a557a2bf21 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:43.046312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2257fe-10c6-4e9f-a0c9-18cad942923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:43.046947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2257fe-10c6-4e9f-a0c9-18cad942923e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:43.047224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2257fe-10c6-4e9f-a0c9-18cad942923e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:43.047496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2257fe-10c6-4e9f-a0c9-18cad942923e None admin] PATH_INFO: `/v3/roles/f948e6f6ee4f42de92195aee35b05641` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:43.048881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2257fe-10c6-4e9f-a0c9-18cad942923e None admin] RBAC: Authorizing `identity:get_role(role_id=f948e6f6ee4f42de92195aee35b05641)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:43.050255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2257fe-10c6-4e9f-a0c9-18cad942923e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:43.052554 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 428/852] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:30:42 2022] GET /identity/v3/roles/f948e6f6ee4f42de92195aee35b05641 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:43.058296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17a2e2-6713-47cc-b8e5-72a483c0582a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:43.110951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:43.163883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17a2e2-6713-47cc-b8e5-72a483c0582a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:43.197959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c67528-dc8c-4638-bc00-6cb4938e5491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:43.198646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c67528-dc8c-4638-bc00-6cb4938e5491 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:43.198922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c67528-dc8c-4638-bc00-6cb4938e5491 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:43.199183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c67528-dc8c-4638-bc00-6cb4938e5491 None admin] PATH_INFO: `/v3/users/889a2b61eb854d58a09982c1e6c5c0dc` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:43.200289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c67528-dc8c-4638-bc00-6cb4938e5491 None admin] RBAC: Authorizing `identity:get_user(user_id=889a2b61eb854d58a09982c1e6c5c0dc)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:43.201691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c67528-dc8c-4638-bc00-6cb4938e5491 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:43.203286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 425/853] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:30:43 2022] GET /identity/v3/users/889a2b61eb854d58a09982c1e6c5c0dc => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:43.209872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2257fe-10c6-4e9f-a0c9-18cad942923e None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:43.252079 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:43.311506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2257fe-10c6-4e9f-a0c9-18cad942923e None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:43.346597 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8efee0-da53-4844-8610-4c17d92dc728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:43.347235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8efee0-da53-4844-8610-4c17d92dc728 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:43.347508 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8efee0-da53-4844-8610-4c17d92dc728 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:43.347774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8efee0-da53-4844-8610-4c17d92dc728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:43.415893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8efee0-da53-4844-8610-4c17d92dc728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:43.417261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8efee0-da53-4844-8610-4c17d92dc728 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:43.457683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:43.517872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 429/854] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:30:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:30:43.523959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c67528-dc8c-4638-bc00-6cb4938e5491 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:43.625443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c67528-dc8c-4638-bc00-6cb4938e5491 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:43.643509 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:43.660869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28234a-88d0-479d-baff-e05034f98010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:43.661498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28234a-88d0-479d-baff-e05034f98010 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:43.661774 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28234a-88d0-479d-baff-e05034f98010 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:43.662041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28234a-88d0-479d-baff-e05034f98010 None admin] PATH_INFO: `/v3/projects/fc4f03999e85403bb0fc55c89f6709a9` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:43.663188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d28234a-88d0-479d-baff-e05034f98010 None admin] RBAC: Authorizing `identity:get_project(project_id=fc4f03999e85403bb0fc55c89f6709a9)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:43.664599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d28234a-88d0-479d-baff-e05034f98010 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:43.666819 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 426/855] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:30:43 2022] GET /identity/v3/projects/fc4f03999e85403bb0fc55c89f6709a9 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:43.673220 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8efee0-da53-4844-8610-4c17d92dc728 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:43.774297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8efee0-da53-4844-8610-4c17d92dc728 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:43.808849 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e5fe2-2890-462f-9110-3bb330027bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:43.809515 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e5fe2-2890-462f-9110-3bb330027bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:43.809799 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e5fe2-2890-462f-9110-3bb330027bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:43.810076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e5fe2-2890-462f-9110-3bb330027bf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:43.810878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e5fe2-2890-462f-9110-3bb330027bf5 None admin] RBAC: Adding query filter params (role.id=f948e6f6ee4f42de92195aee35b05641, scope.project.id=fc4f03999e85403bb0fc55c89f6709a9, user.id=889a2b61eb854d58a09982c1e6c5c0dc) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:43.811183 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e5fe2-2890-462f-9110-3bb330027bf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:43.812623 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e5fe2-2890-462f-9110-3bb330027bf5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:43.819319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 430/856] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:30:43 2022] GET /identity/v3/role_assignments?user.id=889a2b61eb854d58a09982c1e6c5c0dc&scope.project.id=fc4f03999e85403bb0fc55c89f6709a9&role.id=f948e6f6ee4f42de92195aee35b05641 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:30:45.985794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ec5da-35da-4f11-a983-f5ce9ca58653 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:45.986354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ec5da-35da-4f11-a983-f5ce9ca58653 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:45.986354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ec5da-35da-4f11-a983-f5ce9ca58653 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:45.987855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 427/857] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:45.992951 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3190ec-69ed-4e6b-b5d7-001ca31d334c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:45.993366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3190ec-69ed-4e6b-b5d7-001ca31d334c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:45.993492 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3190ec-69ed-4e6b-b5d7-001ca31d334c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:46.000645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:46.011755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ca3190ec-69ed-4e6b-b5d7-001ca31d334c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:46.012912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ca3190ec-69ed-4e6b-b5d7-001ca31d334c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:46.079049 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 431/858] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:46.344385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ec5da-35da-4f11-a983-f5ce9ca58653 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:46.445909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ec5da-35da-4f11-a983-f5ce9ca58653 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:46.478377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ffcf2-0623-47a5-a5a1-7828ea5670fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:46.478990 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ffcf2-0623-47a5-a5a1-7828ea5670fa None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:46.479262 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ffcf2-0623-47a5-a5a1-7828ea5670fa None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:46.479528 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ffcf2-0623-47a5-a5a1-7828ea5670fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:46.480607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ffcf2-0623-47a5-a5a1-7828ea5670fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:46.482012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ffcf2-0623-47a5-a5a1-7828ea5670fa None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:46.483733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 428/859] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:46.490037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3190ec-69ed-4e6b-b5d7-001ca31d334c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:46.593609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3190ec-69ed-4e6b-b5d7-001ca31d334c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:46.626333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8cdf4-5928-496c-8634-4b8dd3e30ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:46.626873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8cdf4-5928-496c-8634-4b8dd3e30ef3 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:46.627150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8cdf4-5928-496c-8634-4b8dd3e30ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:46.627419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8cdf4-5928-496c-8634-4b8dd3e30ef3 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:46.628066 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8cdf4-5928-496c-8634-4b8dd3e30ef3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:46.630220 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8cdf4-5928-496c-8634-4b8dd3e30ef3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:46.634745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:46.666215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 432/860] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:30:46 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:48.852009 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f473fe-a212-49bf-b25c-3e25597cfcc7 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:48.852569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f473fe-a212-49bf-b25c-3e25597cfcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:48.852569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f473fe-a212-49bf-b25c-3e25597cfcc7 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:48.853861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 429/861] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:48.858956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f0dd8-db18-43f1-bf63-ca677ccfeb76 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:48.859356 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f0dd8-db18-43f1-bf63-ca677ccfeb76 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:48.859528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f0dd8-db18-43f1-bf63-ca677ccfeb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:48.877871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0d8f0dd8-db18-43f1-bf63-ca677ccfeb76 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:48.879741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0d8f0dd8-db18-43f1-bf63-ca677ccfeb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:48.944210 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 433/862] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:49.215515 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f473fe-a212-49bf-b25c-3e25597cfcc7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:49.271331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:49.314319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f473fe-a212-49bf-b25c-3e25597cfcc7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:49.346979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc951554-6536-41a5-834d-6daf94a3ce7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:49.347690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc951554-6536-41a5-834d-6daf94a3ce7e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:49.347970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc951554-6536-41a5-834d-6daf94a3ce7e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:49.348237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc951554-6536-41a5-834d-6daf94a3ce7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:49.349328 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc951554-6536-41a5-834d-6daf94a3ce7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:49.350771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc951554-6536-41a5-834d-6daf94a3ce7e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:49.352286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 430/863] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:30:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:49.358017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f0dd8-db18-43f1-bf63-ca677ccfeb76 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:49.454367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f0dd8-db18-43f1-bf63-ca677ccfeb76 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:49.470461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:49.488338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db942517-ca6a-40dc-a3f3-ba9f0102c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:49.488929 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db942517-ca6a-40dc-a3f3-ba9f0102c996 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:49.489202 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db942517-ca6a-40dc-a3f3-ba9f0102c996 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:49.489463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db942517-ca6a-40dc-a3f3-ba9f0102c996 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:49.490108 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db942517-ca6a-40dc-a3f3-ba9f0102c996 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:49.491521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db942517-ca6a-40dc-a3f3-ba9f0102c996 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:49.512222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 434/864] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:30:49 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:51.680384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129db425-1ead-4a17-b599-5b37dae8478a None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:51.680912 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129db425-1ead-4a17-b599-5b37dae8478a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:51.680912 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129db425-1ead-4a17-b599-5b37dae8478a None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:51.682237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 431/865] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:51.687345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92371b-f52b-4cf2-8a82-d486f181846a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:51.687744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92371b-f52b-4cf2-8a82-d486f181846a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:51.687901 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92371b-f52b-4cf2-8a82-d486f181846a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:51.706143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ae92371b-f52b-4cf2-8a82-d486f181846a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:51.708017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ae92371b-f52b-4cf2-8a82-d486f181846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:51.777678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 435/866] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:52.042911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129db425-1ead-4a17-b599-5b37dae8478a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:52.140216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129db425-1ead-4a17-b599-5b37dae8478a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:52.161850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:52.173327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b67fa-a289-44a2-9568-373e96dabbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:52.173928 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b67fa-a289-44a2-9568-373e96dabbf4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:52.174222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b67fa-a289-44a2-9568-373e96dabbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:52.174487 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b67fa-a289-44a2-9568-373e96dabbf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:52.184714 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b67fa-a289-44a2-9568-373e96dabbf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-037b67fa-a289-44a2-9568-373e96dabbf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:30:52.189387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:52.190527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:30:52.191387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 432/867] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:30:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:52.196400 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92371b-f52b-4cf2-8a82-d486f181846a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:52.293131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92371b-f52b-4cf2-8a82-d486f181846a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:52.294563 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:52.328285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d63f76-9d73-46ec-b2c3-f7fe9c09a5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:52.328885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63f76-9d73-46ec-b2c3-f7fe9c09a5af None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:52.329161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63f76-9d73-46ec-b2c3-f7fe9c09a5af None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:52.329428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d63f76-9d73-46ec-b2c3-f7fe9c09a5af None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:52.330079 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d63f76-9d73-46ec-b2c3-f7fe9c09a5af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:52.330403 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d63f76-9d73-46ec-b2c3-f7fe9c09a5af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:52.331832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d63f76-9d73-46ec-b2c3-f7fe9c09a5af None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:52.339533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 436/868] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:30:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:52.345795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b67fa-a289-44a2-9568-373e96dabbf4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:52.441609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b67fa-a289-44a2-9568-373e96dabbf4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:52.474861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d8f93-4932-48cb-90d1-f436160d262c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:52.475474 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d8f93-4932-48cb-90d1-f436160d262c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:52.475751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d8f93-4932-48cb-90d1-f436160d262c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:52.476018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d8f93-4932-48cb-90d1-f436160d262c None admin] PATH_INFO: `/v3/users/533aa73e9fd04bec9d9c30bf9026a2d7` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:52.488553 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d8f93-4932-48cb-90d1-f436160d262c None admin] RBAC: Authorizing `identity:get_user(user_id=533aa73e9fd04bec9d9c30bf9026a2d7)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:52.489966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d8f93-4932-48cb-90d1-f436160d262c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:52.491560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 433/869] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:30:52 2022] GET /identity/v3/users/533aa73e9fd04bec9d9c30bf9026a2d7 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:52.497167 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d63f76-9d73-46ec-b2c3-f7fe9c09a5af None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:52.593286 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d63f76-9d73-46ec-b2c3-f7fe9c09a5af None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:52.625224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3ad36-f64f-4def-a963-201984554a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:52.625837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3ad36-f64f-4def-a963-201984554a12 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:52.626110 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3ad36-f64f-4def-a963-201984554a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:52.626401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3ad36-f64f-4def-a963-201984554a12 None admin] PATH_INFO: `/v3/projects/9227b0d695fb49e2b72588a8c78063af` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:52.627480 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3ad36-f64f-4def-a963-201984554a12 None admin] RBAC: Authorizing `identity:get_project(project_id=9227b0d695fb49e2b72588a8c78063af)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:52.629026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3ad36-f64f-4def-a963-201984554a12 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:52.630635 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 437/870] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:30:52 2022] GET /identity/v3/projects/9227b0d695fb49e2b72588a8c78063af => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:52.636477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d8f93-4932-48cb-90d1-f436160d262c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:52.677749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:52.734133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d8f93-4932-48cb-90d1-f436160d262c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:52.766562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21a9f2-1e7f-4d90-8636-a4757a6b79dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:52.767201 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21a9f2-1e7f-4d90-8636-a4757a6b79dd None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:52.767480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21a9f2-1e7f-4d90-8636-a4757a6b79dd None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:52.767749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21a9f2-1e7f-4d90-8636-a4757a6b79dd None admin] PATH_INFO: `/v3/projects/9227b0d695fb49e2b72588a8c78063af/users/533aa73e9fd04bec9d9c30bf9026a2d7/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:52.770379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21a9f2-1e7f-4d90-8636-a4757a6b79dd None admin] RBAC: Authorizing `identity:create_grant(project_id=9227b0d695fb49e2b72588a8c78063af, user_id=533aa73e9fd04bec9d9c30bf9026a2d7, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:52.771791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21a9f2-1e7f-4d90-8636-a4757a6b79dd None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:52.784238 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 434/871] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:30:52 2022] PUT /identity/v3/projects/9227b0d695fb49e2b72588a8c78063af/users/533aa73e9fd04bec9d9c30bf9026a2d7/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:30:54.923760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42b0b9-e3f3-4a0a-9eaf-152a7036a3a3 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:54.924387 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42b0b9-e3f3-4a0a-9eaf-152a7036a3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:54.924387 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42b0b9-e3f3-4a0a-9eaf-152a7036a3a3 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:54.926115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 438/872] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:54.931414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a233d4-73de-410a-a4d9-9e8c582681a7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:54.931840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a233d4-73de-410a-a4d9-9e8c582681a7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:54.931939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a233d4-73de-410a-a4d9-9e8c582681a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:54.950372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-76a233d4-73de-410a-a4d9-9e8c582681a7 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:54.951531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-76a233d4-73de-410a-a4d9-9e8c582681a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:55.015613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 435/873] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:55.316748 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42b0b9-e3f3-4a0a-9eaf-152a7036a3a3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:55.347642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:55.416517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42b0b9-e3f3-4a0a-9eaf-152a7036a3a3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:55.448667 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f18d4-27cc-47ec-9408-5408332f03d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:55.449281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f18d4-27cc-47ec-9408-5408332f03d1 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:55.449559 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f18d4-27cc-47ec-9408-5408332f03d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:55.449828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f18d4-27cc-47ec-9408-5408332f03d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:55.460874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f18d4-27cc-47ec-9408-5408332f03d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-356f18d4-27cc-47ec-9408-5408332f03d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:30:55.465524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:30:55.466631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:30:55.467519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 439/874] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:30:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:30:55.473321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a233d4-73de-410a-a4d9-9e8c582681a7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:55.501327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:55.569885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a233d4-73de-410a-a4d9-9e8c582681a7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:55.603264 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc95e21-4b9b-4a4c-9c7c-a0ab2d4c6491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:55.603857 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc95e21-4b9b-4a4c-9c7c-a0ab2d4c6491 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:55.604133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc95e21-4b9b-4a4c-9c7c-a0ab2d4c6491 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:55.604398 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc95e21-4b9b-4a4c-9c7c-a0ab2d4c6491 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:55.605046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc95e21-4b9b-4a4c-9c7c-a0ab2d4c6491 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:55.605343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc95e21-4b9b-4a4c-9c7c-a0ab2d4c6491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:55.606725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc95e21-4b9b-4a4c-9c7c-a0ab2d4c6491 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:55.614834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 436/875] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:30:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:55.620746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f18d4-27cc-47ec-9408-5408332f03d1 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:55.711635 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:55.720437 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f18d4-27cc-47ec-9408-5408332f03d1 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:55.752925 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a15d2b-bdcb-4854-b508-260adc65fedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:55.753569 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a15d2b-bdcb-4854-b508-260adc65fedd None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:55.753938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a15d2b-bdcb-4854-b508-260adc65fedd None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:55.754237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a15d2b-bdcb-4854-b508-260adc65fedd None admin] PATH_INFO: `/v3/users/533aa73e9fd04bec9d9c30bf9026a2d7` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:55.755339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a15d2b-bdcb-4854-b508-260adc65fedd None admin] RBAC: Authorizing `identity:get_user(user_id=533aa73e9fd04bec9d9c30bf9026a2d7)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:55.756741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a15d2b-bdcb-4854-b508-260adc65fedd None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:55.758314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 440/876] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:30:55 2022] GET /identity/v3/users/533aa73e9fd04bec9d9c30bf9026a2d7 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:55.764050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc95e21-4b9b-4a4c-9c7c-a0ab2d4c6491 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:55.852832 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:55.861592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc95e21-4b9b-4a4c-9c7c-a0ab2d4c6491 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:55.893592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa3034-e02b-4896-ba9a-42343ed322d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:55.894920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa3034-e02b-4896-ba9a-42343ed322d7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:55.895191 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa3034-e02b-4896-ba9a-42343ed322d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:55.895455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa3034-e02b-4896-ba9a-42343ed322d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:55.959434 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa3034-e02b-4896-ba9a-42343ed322d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:55.960857 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa3034-e02b-4896-ba9a-42343ed322d7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:56.044743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:56.054360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 437/877] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:30:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:30:56.060301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a15d2b-bdcb-4854-b508-260adc65fedd None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:56.157155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a15d2b-bdcb-4854-b508-260adc65fedd None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:56.189209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b8f49-1f99-49ee-9d6e-03ec3b7157c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:56.189819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b8f49-1f99-49ee-9d6e-03ec3b7157c0 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:56.190101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b8f49-1f99-49ee-9d6e-03ec3b7157c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:56.190394 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b8f49-1f99-49ee-9d6e-03ec3b7157c0 None admin] PATH_INFO: `/v3/projects/9227b0d695fb49e2b72588a8c78063af` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:56.192004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b8f49-1f99-49ee-9d6e-03ec3b7157c0 None admin] RBAC: Authorizing `identity:get_project(project_id=9227b0d695fb49e2b72588a8c78063af)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:56.193414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b8f49-1f99-49ee-9d6e-03ec3b7157c0 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:56.195021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 441/878] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:30:56 2022] GET /identity/v3/projects/9227b0d695fb49e2b72588a8c78063af => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:56.201050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa3034-e02b-4896-ba9a-42343ed322d7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:56.295574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa3034-e02b-4896-ba9a-42343ed322d7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:56.328600 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda68f4-011f-4add-a118-0d7169f905c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:56.329203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda68f4-011f-4add-a118-0d7169f905c8 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:56.329483 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda68f4-011f-4add-a118-0d7169f905c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:56.329749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda68f4-011f-4add-a118-0d7169f905c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:56.330498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda68f4-011f-4add-a118-0d7169f905c8 None admin] RBAC: Adding query filter params (role.id=8b32d66a3cb3483c8ede361ad8637f66, scope.project.id=9227b0d695fb49e2b72588a8c78063af, user.id=533aa73e9fd04bec9d9c30bf9026a2d7) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:56.330801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda68f4-011f-4add-a118-0d7169f905c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:56.332390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda68f4-011f-4add-a118-0d7169f905c8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:56.337945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 438/879] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:30:56 2022] GET /identity/v3/role_assignments?user.id=533aa73e9fd04bec9d9c30bf9026a2d7&scope.project.id=9227b0d695fb49e2b72588a8c78063af&role.id=8b32d66a3cb3483c8ede361ad8637f66 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:58.640740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b35c7-6a7b-4111-85d8-a3355be2986f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:58.641296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b35c7-6a7b-4111-85d8-a3355be2986f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:58.641296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b35c7-6a7b-4111-85d8-a3355be2986f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:58.642632 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 442/880] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:58.648179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5466b8-56dd-489f-8244-b1a7130d24ee None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:58.648618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5466b8-56dd-489f-8244-b1a7130d24ee None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:58.648717 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5466b8-56dd-489f-8244-b1a7130d24ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:58.666746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0b5466b8-56dd-489f-8244-b1a7130d24ee None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:58.667911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0b5466b8-56dd-489f-8244-b1a7130d24ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:58.716367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:58.732800 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 439/881] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:30:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:30:59.004701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b35c7-6a7b-4111-85d8-a3355be2986f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:59.033004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:59.105808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b35c7-6a7b-4111-85d8-a3355be2986f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:59.138145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b5386-1d41-4d4b-843e-d0f749963360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:59.138760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b5386-1d41-4d4b-843e-d0f749963360 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:59.139033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b5386-1d41-4d4b-843e-d0f749963360 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:59.139297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b5386-1d41-4d4b-843e-d0f749963360 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:59.139949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b5386-1d41-4d4b-843e-d0f749963360 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:59.141282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b5386-1d41-4d4b-843e-d0f749963360 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:59.156490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 443/882] 10.222.0.20 () {60 vars in 1138 bytes} [Mon Oct 24 13:30:59 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:31:01.343217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d5a70-e157-406c-b6cd-a6f166a276f0 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:01.343738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d5a70-e157-406c-b6cd-a6f166a276f0 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:01.343738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d5a70-e157-406c-b6cd-a6f166a276f0 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:01.345091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 440/883] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:31:01.351153 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a239b8-352d-47f9-8b63-2b1e55134b82 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:01.351590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a239b8-352d-47f9-8b63-2b1e55134b82 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:01.351680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a239b8-352d-47f9-8b63-2b1e55134b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:01.369998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e7a239b8-352d-47f9-8b63-2b1e55134b82 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:31:01.371234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e7a239b8-352d-47f9-8b63-2b1e55134b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:01.435956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 444/884] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:31:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:31:01.704197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d5a70-e157-406c-b6cd-a6f166a276f0 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:01.806644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d5a70-e157-406c-b6cd-a6f166a276f0 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:01.841972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d673f8-4ade-40a3-a84a-beca137844c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:01.842624 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d673f8-4ade-40a3-a84a-beca137844c2 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:01.842904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d673f8-4ade-40a3-a84a-beca137844c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:01.843175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d673f8-4ade-40a3-a84a-beca137844c2 None admin] PATH_INFO: `/v3/domains/b0fe99d5c28f4b688ebd84e0dc0cc32d` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:01.853303 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d673f8-4ade-40a3-a84a-beca137844c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b0fe99d5c28f4b688ebd84e0dc0cc32d)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:01.854761 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d673f8-4ade-40a3-a84a-beca137844c2 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:01.856314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 441/885] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Oct 24 13:31:01 2022] GET /identity/v3/domains/b0fe99d5c28f4b688ebd84e0dc0cc32d => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:31:01.862144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a239b8-352d-47f9-8b63-2b1e55134b82 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:01.880221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:01.959239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a239b8-352d-47f9-8b63-2b1e55134b82 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:01.991229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f032fe6-c0fd-4c31-b831-b92073f76b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:01.991827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032fe6-c0fd-4c31-b831-b92073f76b78 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:01.992102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032fe6-c0fd-4c31-b831-b92073f76b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:01.992367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032fe6-c0fd-4c31-b831-b92073f76b78 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:01.993014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f032fe6-c0fd-4c31-b831-b92073f76b78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:01.994444 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f032fe6-c0fd-4c31-b831-b92073f76b78 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:02.020976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 445/886] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:31:01 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:31:04.180121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da6732-ad07-4677-aebe-b690d15e520f None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:04.180649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da6732-ad07-4677-aebe-b690d15e520f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:04.180649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da6732-ad07-4677-aebe-b690d15e520f None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:04.182029 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 442/887] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:31:04.187400 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faeb97-4f91-4ba1-82be-5eaf7237a64b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:04.187807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faeb97-4f91-4ba1-82be-5eaf7237a64b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:04.187943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faeb97-4f91-4ba1-82be-5eaf7237a64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:04.207396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f9faeb97-4f91-4ba1-82be-5eaf7237a64b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:31:04.208634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f9faeb97-4f91-4ba1-82be-5eaf7237a64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:04.268319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:04.273715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 446/888] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:31:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:31:04.543207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da6732-ad07-4677-aebe-b690d15e520f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:04.588640 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:04.640566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da6732-ad07-4677-aebe-b690d15e520f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:04.673368 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725a242-3e99-4186-b414-5a152baa3eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:04.673985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725a242-3e99-4186-b414-5a152baa3eec None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:04.674288 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725a242-3e99-4186-b414-5a152baa3eec None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:04.674556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725a242-3e99-4186-b414-5a152baa3eec None admin] PATH_INFO: `/v3/domains/b0fe99d5c28f4b688ebd84e0dc0cc32d` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:04.675899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725a242-3e99-4186-b414-5a152baa3eec None admin] RBAC: Authorizing `identity:get_domain(domain_id=b0fe99d5c28f4b688ebd84e0dc0cc32d)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:04.677295 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725a242-3e99-4186-b414-5a152baa3eec None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:04.678844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 443/889] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Oct 24 13:31:04 2022] GET /identity/v3/domains/b0fe99d5c28f4b688ebd84e0dc0cc32d => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:31:04.684509 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9faeb97-4f91-4ba1-82be-5eaf7237a64b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:04.780354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9faeb97-4f91-4ba1-82be-5eaf7237a64b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:04.813683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04832b1a-f0a0-4139-8af6-5bdfb7a118fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:04.814301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832b1a-f0a0-4139-8af6-5bdfb7a118fe None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:04.814583 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832b1a-f0a0-4139-8af6-5bdfb7a118fe None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:04.814853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832b1a-f0a0-4139-8af6-5bdfb7a118fe None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:04.816041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04832b1a-f0a0-4139-8af6-5bdfb7a118fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:04.817535 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04832b1a-f0a0-4139-8af6-5bdfb7a118fe None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:04.839301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 447/890] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:31:04 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:31:06.989920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907876cf-b480-4c83-9392-16ed2f7a6b97 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:06.990513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907876cf-b480-4c83-9392-16ed2f7a6b97 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:06.990513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907876cf-b480-4c83-9392-16ed2f7a6b97 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:06.991792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 444/891] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:31:06.996843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2829e0-723b-451d-ad28-9a18c02d3fcb None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:06.997229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2829e0-723b-451d-ad28-9a18c02d3fcb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:06.997396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2829e0-723b-451d-ad28-9a18c02d3fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:07.015258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0e2829e0-723b-451d-ad28-9a18c02d3fcb None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:31:07.016424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0e2829e0-723b-451d-ad28-9a18c02d3fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:07.065171 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:07.082666 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 448/892] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:31:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:31:07.346836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907876cf-b480-4c83-9392-16ed2f7a6b97 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:07.445449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907876cf-b480-4c83-9392-16ed2f7a6b97 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:07.459287 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:07.478608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85f7fa-a25d-40d4-a7f1-ea990747e577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:07.479222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f7fa-a25d-40d4-a7f1-ea990747e577 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:07.479497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f7fa-a25d-40d4-a7f1-ea990747e577 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:07.479763 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f7fa-a25d-40d4-a7f1-ea990747e577 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:07.489919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85f7fa-a25d-40d4-a7f1-ea990747e577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-7d85f7fa-a25d-40d4-a7f1-ea990747e577 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:31:07.494674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:31:07.495769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:31:07.496981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 445/893] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:31:07.501667 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2829e0-723b-451d-ad28-9a18c02d3fcb None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:07.596767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2829e0-723b-451d-ad28-9a18c02d3fcb None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:07.629884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d6096-2114-4da9-a5a8-e4128193fefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:07.630444 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d6096-2114-4da9-a5a8-e4128193fefd None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:07.630728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d6096-2114-4da9-a5a8-e4128193fefd None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:07.630998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d6096-2114-4da9-a5a8-e4128193fefd None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:07.631650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d6096-2114-4da9-a5a8-e4128193fefd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:31:07.631948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d6096-2114-4da9-a5a8-e4128193fefd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:07.633296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d6096-2114-4da9-a5a8-e4128193fefd None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:07.640949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 449/894] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:31:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:31:07.646992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85f7fa-a25d-40d4-a7f1-ea990747e577 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:07.743401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85f7fa-a25d-40d4-a7f1-ea990747e577 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:07.776104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d431504-549a-4af9-954d-eb28541d3c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:07.776709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d431504-549a-4af9-954d-eb28541d3c8d None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:07.776988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d431504-549a-4af9-954d-eb28541d3c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:07.777258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d431504-549a-4af9-954d-eb28541d3c8d None admin] PATH_INFO: `/v3/users/c968367ef20647249890ecc3b29c8e48` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:07.789680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d431504-549a-4af9-954d-eb28541d3c8d None admin] RBAC: Authorizing `identity:get_user(user_id=c968367ef20647249890ecc3b29c8e48)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:07.791404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d431504-549a-4af9-954d-eb28541d3c8d None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:07.792948 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 446/895] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:31:07 2022] GET /identity/v3/users/c968367ef20647249890ecc3b29c8e48 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:31:07.798588 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d6096-2114-4da9-a5a8-e4128193fefd None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:07.848361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:07.898497 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d6096-2114-4da9-a5a8-e4128193fefd None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:07.931534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed09f5-6365-4cb3-887b-9382455e26b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:07.932149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed09f5-6365-4cb3-887b-9382455e26b5 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:07.932426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed09f5-6365-4cb3-887b-9382455e26b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:07.932694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed09f5-6365-4cb3-887b-9382455e26b5 None admin] PATH_INFO: `/v3/projects/ae2233e05d8e433daf0889873e81e9cd` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:07.933786 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed09f5-6365-4cb3-887b-9382455e26b5 None admin] RBAC: Authorizing `identity:get_project(project_id=ae2233e05d8e433daf0889873e81e9cd)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:07.935224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed09f5-6365-4cb3-887b-9382455e26b5 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:07.936823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 450/896] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:31:07 2022] GET /identity/v3/projects/ae2233e05d8e433daf0889873e81e9cd => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:31:07.943204 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d431504-549a-4af9-954d-eb28541d3c8d None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:07.973192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:08.038853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d431504-549a-4af9-954d-eb28541d3c8d None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:08.071442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c1111-dbc6-4f86-9746-136ae8c084ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:08.072078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c1111-dbc6-4f86-9746-136ae8c084ab None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:08.072357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c1111-dbc6-4f86-9746-136ae8c084ab None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:08.072626 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c1111-dbc6-4f86-9746-136ae8c084ab None admin] PATH_INFO: `/v3/projects/ae2233e05d8e433daf0889873e81e9cd/users/c968367ef20647249890ecc3b29c8e48/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:08.074503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c1111-dbc6-4f86-9746-136ae8c084ab None admin] RBAC: Authorizing `identity:create_grant(project_id=ae2233e05d8e433daf0889873e81e9cd, user_id=c968367ef20647249890ecc3b29c8e48, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:08.075900 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c1111-dbc6-4f86-9746-136ae8c084ab None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:08.088032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 447/897] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:31:07 2022] PUT /identity/v3/projects/ae2233e05d8e433daf0889873e81e9cd/users/c968367ef20647249890ecc3b29c8e48/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:31:10.195129 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83917bef-af34-4316-8947-6da5c9df24a9 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:10.195706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83917bef-af34-4316-8947-6da5c9df24a9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:10.195706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83917bef-af34-4316-8947-6da5c9df24a9 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:10.196989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 451/898] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:31:10.202198 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aabbeb-38e4-487b-919f-51bcafdfb08b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:10.202629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aabbeb-38e4-487b-919f-51bcafdfb08b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:10.202765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aabbeb-38e4-487b-919f-51bcafdfb08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:10.221314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-49aabbeb-38e4-487b-919f-51bcafdfb08b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:31:10.223236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-49aabbeb-38e4-487b-919f-51bcafdfb08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:10.286998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 448/899] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:31:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:31:10.561044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83917bef-af34-4316-8947-6da5c9df24a9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:10.661970 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83917bef-af34-4316-8947-6da5c9df24a9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:10.677554 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:10.696631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de20ef-9e1b-4d0c-8525-36112dfb416d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:10.697258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de20ef-9e1b-4d0c-8525-36112dfb416d None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:10.697537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de20ef-9e1b-4d0c-8525-36112dfb416d None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:10.697809 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de20ef-9e1b-4d0c-8525-36112dfb416d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:10.708446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de20ef-9e1b-4d0c-8525-36112dfb416d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-68de20ef-9e1b-4d0c-8525-36112dfb416d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:31:10.713199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:31:10.714253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:31:10.715496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 452/900] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:31:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:31:10.720289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aabbeb-38e4-487b-919f-51bcafdfb08b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:10.743818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:10.817978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aabbeb-38e4-487b-919f-51bcafdfb08b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:10.852985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000a373-2c75-4f16-bb92-f2c41dc9f445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:10.853573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000a373-2c75-4f16-bb92-f2c41dc9f445 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:10.853844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000a373-2c75-4f16-bb92-f2c41dc9f445 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:10.854107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000a373-2c75-4f16-bb92-f2c41dc9f445 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:10.854782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000a373-2c75-4f16-bb92-f2c41dc9f445 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:31:10.855076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000a373-2c75-4f16-bb92-f2c41dc9f445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:10.856415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000a373-2c75-4f16-bb92-f2c41dc9f445 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:10.864327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 449/901] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:31:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:31:10.870288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de20ef-9e1b-4d0c-8525-36112dfb416d None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:10.966698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de20ef-9e1b-4d0c-8525-36112dfb416d None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:10.998126 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aceeeb-9476-4e1c-be05-64db312f55b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:10.998761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aceeeb-9476-4e1c-be05-64db312f55b2 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:10.999036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aceeeb-9476-4e1c-be05-64db312f55b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:10.999307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aceeeb-9476-4e1c-be05-64db312f55b2 None admin] PATH_INFO: `/v3/users/c968367ef20647249890ecc3b29c8e48` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:11.000396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aceeeb-9476-4e1c-be05-64db312f55b2 None admin] RBAC: Authorizing `identity:get_user(user_id=c968367ef20647249890ecc3b29c8e48)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:11.001805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aceeeb-9476-4e1c-be05-64db312f55b2 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:11.003902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 453/902] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Oct 24 13:31:10 2022] GET /identity/v3/users/c968367ef20647249890ecc3b29c8e48 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:31:11.009778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000a373-2c75-4f16-bb92-f2c41dc9f445 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:11.094534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:11.111442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000a373-2c75-4f16-bb92-f2c41dc9f445 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:11.144593 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e6f6c-0bb7-4207-90f5-a5ce2d373cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:11.145210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e6f6c-0bb7-4207-90f5-a5ce2d373cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:11.145481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e6f6c-0bb7-4207-90f5-a5ce2d373cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:11.145740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e6f6c-0bb7-4207-90f5-a5ce2d373cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:11.210856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e6f6c-0bb7-4207-90f5-a5ce2d373cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:11.212307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e6f6c-0bb7-4207-90f5-a5ce2d373cb7 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:11.290667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:11.307019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 450/903] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:31:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:31:11.312884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aceeeb-9476-4e1c-be05-64db312f55b2 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:11.358004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:11.411498 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aceeeb-9476-4e1c-be05-64db312f55b2 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:11.444651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea4959-c92c-4883-a692-cb59f933d9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:11.445264 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea4959-c92c-4883-a692-cb59f933d9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:11.445536 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea4959-c92c-4883-a692-cb59f933d9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:11.445802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea4959-c92c-4883-a692-cb59f933d9f7 None admin] PATH_INFO: `/v3/projects/ae2233e05d8e433daf0889873e81e9cd` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:11.446903 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea4959-c92c-4883-a692-cb59f933d9f7 None admin] RBAC: Authorizing `identity:get_project(project_id=ae2233e05d8e433daf0889873e81e9cd)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:11.448303 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea4959-c92c-4883-a692-cb59f933d9f7 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:11.449881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 454/904] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Oct 24 13:31:11 2022] GET /identity/v3/projects/ae2233e05d8e433daf0889873e81e9cd => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:31:11.455964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e6f6c-0bb7-4207-90f5-a5ce2d373cb7 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:11.552119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e6f6c-0bb7-4207-90f5-a5ce2d373cb7 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:11.585190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f38bc-73e2-40d2-95b7-5504f5b8a397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:11.585786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f38bc-73e2-40d2-95b7-5504f5b8a397 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:11.586064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f38bc-73e2-40d2-95b7-5504f5b8a397 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:11.586363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f38bc-73e2-40d2-95b7-5504f5b8a397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:11.587076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f38bc-73e2-40d2-95b7-5504f5b8a397 None admin] RBAC: Adding query filter params (role.id=8b32d66a3cb3483c8ede361ad8637f66, scope.project.id=ae2233e05d8e433daf0889873e81e9cd, user.id=c968367ef20647249890ecc3b29c8e48) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:31:11.587381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f38bc-73e2-40d2-95b7-5504f5b8a397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:11.588793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f38bc-73e2-40d2-95b7-5504f5b8a397 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:11.594351 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 451/905] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:31:11 2022] GET /identity/v3/role_assignments?user.id=c968367ef20647249890ecc3b29c8e48&scope.project.id=ae2233e05d8e433daf0889873e81e9cd&role.id=8b32d66a3cb3483c8ede361ad8637f66 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:31:21.440640 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=44 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:21.591802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=76 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:07.571832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ced4c6-bcea-4a05-9b5b-7252e9b925c4 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:07.572470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ced4c6-bcea-4a05-9b5b-7252e9b925c4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:07.572470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ced4c6-bcea-4a05-9b5b-7252e9b925c4 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:07.573887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 455/906] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:32:07.582206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152658d-d53f-4547-86ac-a04a2f9c80aa None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:07.582206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152658d-d53f-4547-86ac-a04a2f9c80aa None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:07.582206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152658d-d53f-4547-86ac-a04a2f9c80aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:07.600688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9152658d-d53f-4547-86ac-a04a2f9c80aa None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:07.601923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9152658d-d53f-4547-86ac-a04a2f9c80aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:07.684987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 452/907] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:32:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:32:07.997109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ced4c6-bcea-4a05-9b5b-7252e9b925c4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:08.137776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ced4c6-bcea-4a05-9b5b-7252e9b925c4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:08.181102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93f84d-1597-4a5d-ad35-3970a2b59628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:08.182176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93f84d-1597-4a5d-ad35-3970a2b59628 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:08.182421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93f84d-1597-4a5d-ad35-3970a2b59628 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:08.182767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93f84d-1597-4a5d-ad35-3970a2b59628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:08.194836 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93f84d-1597-4a5d-ad35-3970a2b59628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:08.202056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93f84d-1597-4a5d-ad35-3970a2b59628 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-0a93f84d-1597-4a5d-ad35-3970a2b59628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:32:08.220575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:08.222722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:32:08.223841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:32:08.223841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:08.223841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:32:08.225592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 456/908] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:32:08.232410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152658d-d53f-4547-86ac-a04a2f9c80aa None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:08.373798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152658d-d53f-4547-86ac-a04a2f9c80aa None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:08.411133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:08.418806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724d859-af18-4eb3-90d3-b7c8446ac5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:08.419461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724d859-af18-4eb3-90d3-b7c8446ac5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:08.419902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724d859-af18-4eb3-90d3-b7c8446ac5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:08.420108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724d859-af18-4eb3-90d3-b7c8446ac5f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:08.421063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724d859-af18-4eb3-90d3-b7c8446ac5f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:08.421394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724d859-af18-4eb3-90d3-b7c8446ac5f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:08.422974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724d859-af18-4eb3-90d3-b7c8446ac5f1 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:08.439390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 453/909] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:08.449755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93f84d-1597-4a5d-ad35-3970a2b59628 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:08.519020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:08.582357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93f84d-1597-4a5d-ad35-3970a2b59628 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:08.630651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d61d8-3159-445c-bbfa-f7f732300fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:08.631159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d61d8-3159-445c-bbfa-f7f732300fba None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:08.631499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d61d8-3159-445c-bbfa-f7f732300fba None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:08.631713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d61d8-3159-445c-bbfa-f7f732300fba None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:08.632334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d61d8-3159-445c-bbfa-f7f732300fba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:08.637520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d61d8-3159-445c-bbfa-f7f732300fba None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:08.668981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 457/910] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Oct 24 13:32:08 2022] POST /identity/v3/users => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:32:11.056012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c30a7-1417-48aa-a980-800b1bc3e34e None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:11.056571 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c30a7-1417-48aa-a980-800b1bc3e34e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:11.056571 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c30a7-1417-48aa-a980-800b1bc3e34e None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:11.058028 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 454/911] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:32:11.063307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511859b-0361-4888-9309-f9f4bd8276f1 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:11.064683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511859b-0361-4888-9309-f9f4bd8276f1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:11.064683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511859b-0361-4888-9309-f9f4bd8276f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:11.083151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7511859b-0361-4888-9309-f9f4bd8276f1 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:11.084357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7511859b-0361-4888-9309-f9f4bd8276f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:11.160585 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 458/912] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:32:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:32:11.444170 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c30a7-1417-48aa-a980-800b1bc3e34e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:11.556566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c30a7-1417-48aa-a980-800b1bc3e34e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:11.600899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdf41e-9be3-4bef-b8d4-6ef556ef3760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:11.601561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdf41e-9be3-4bef-b8d4-6ef556ef3760 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:11.601906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdf41e-9be3-4bef-b8d4-6ef556ef3760 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:11.602139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdf41e-9be3-4bef-b8d4-6ef556ef3760 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:11.613375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdf41e-9be3-4bef-b8d4-6ef556ef3760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6efdf41e-9be3-4bef-b8d4-6ef556ef3760 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:32:11.618393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:11.619494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:11.620693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 455/913] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:32:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:11.625692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511859b-0361-4888-9309-f9f4bd8276f1 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:11.671398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:11.745399 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511859b-0361-4888-9309-f9f4bd8276f1 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:11.781278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29014151-cd62-4995-b5b4-b0b153353c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:11.782065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29014151-cd62-4995-b5b4-b0b153353c46 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:11.782580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29014151-cd62-4995-b5b4-b0b153353c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:11.782932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29014151-cd62-4995-b5b4-b0b153353c46 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:11.784004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29014151-cd62-4995-b5b4-b0b153353c46 None admin] RBAC: Adding query filter params (name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:11.784464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29014151-cd62-4995-b5b4-b0b153353c46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:11.785973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29014151-cd62-4995-b5b4-b0b153353c46 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:11.800869 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 459/914] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:32:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:11.802335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdf41e-9be3-4bef-b8d4-6ef556ef3760 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:11.865535 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:11.932763 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdf41e-9be3-4bef-b8d4-6ef556ef3760 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:11.972733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8b04f-2c4e-4ea7-bc09-e0a61968482e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:11.972733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8b04f-2c4e-4ea7-bc09-e0a61968482e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:11.972733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8b04f-2c4e-4ea7-bc09-e0a61968482e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:11.972733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8b04f-2c4e-4ea7-bc09-e0a61968482e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:11.986660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8b04f-2c4e-4ea7-bc09-e0a61968482e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:11.988774 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8b04f-2c4e-4ea7-bc09-e0a61968482e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-7df8b04f-2c4e-4ea7-bc09-e0a61968482e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:32:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:12.007733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:32:12.008670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:32:12.008670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:12.008670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:12.014214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 456/915] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:32:12.014392 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29014151-cd62-4995-b5b4-b0b153353c46 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:12.135364 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:12.138576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29014151-cd62-4995-b5b4-b0b153353c46 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:12.190247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b72c0-461e-41a3-aa20-4554ef09527f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:12.191026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b72c0-461e-41a3-aa20-4554ef09527f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:12.191362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b72c0-461e-41a3-aa20-4554ef09527f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:12.191674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b72c0-461e-41a3-aa20-4554ef09527f None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:12.192379 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b72c0-461e-41a3-aa20-4554ef09527f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:12.192678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b72c0-461e-41a3-aa20-4554ef09527f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:12.194128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b72c0-461e-41a3-aa20-4554ef09527f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:12.211194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 460/916] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:12.214212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8b04f-2c4e-4ea7-bc09-e0a61968482e None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:12.318468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8b04f-2c4e-4ea7-bc09-e0a61968482e None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:12.320400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:12.356685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309bec0-37ca-43dd-8c9e-ae2da16c5d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:12.357346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309bec0-37ca-43dd-8c9e-ae2da16c5d36 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:12.357651 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309bec0-37ca-43dd-8c9e-ae2da16c5d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:12.357921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309bec0-37ca-43dd-8c9e-ae2da16c5d36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:12.367792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309bec0-37ca-43dd-8c9e-ae2da16c5d36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:12.369414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309bec0-37ca-43dd-8c9e-ae2da16c5d36 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-6309bec0-37ca-43dd-8c9e-ae2da16c5d36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:32:12.381886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:12.383083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:12.384160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:12.384160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:12.384160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:32:12.384160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:32:12.384160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:32:12.384160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:32:12.384160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:32:12.384160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:32:12.384160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:32:12.384160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:12.384160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 457/917] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:12.388729 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b72c0-461e-41a3-aa20-4554ef09527f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:12.498965 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b72c0-461e-41a3-aa20-4554ef09527f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:12.536701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237993a-0133-4db9-82ab-23cff94bc414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:12.537487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237993a-0133-4db9-82ab-23cff94bc414 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:12.537639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237993a-0133-4db9-82ab-23cff94bc414 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:12.537816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237993a-0133-4db9-82ab-23cff94bc414 None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:12.538773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237993a-0133-4db9-82ab-23cff94bc414 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:12.539103 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237993a-0133-4db9-82ab-23cff94bc414 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:12.540925 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237993a-0133-4db9-82ab-23cff94bc414 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:12.562128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 461/918] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Oct 24 13:32:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:12.568635 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309bec0-37ca-43dd-8c9e-ae2da16c5d36 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:12.681575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309bec0-37ca-43dd-8c9e-ae2da16c5d36 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:12.719695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd830ad1-83d6-4ff2-9edb-7f6cfc309238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:12.721107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd830ad1-83d6-4ff2-9edb-7f6cfc309238 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:12.721470 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd830ad1-83d6-4ff2-9edb-7f6cfc309238 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:12.721930 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd830ad1-83d6-4ff2-9edb-7f6cfc309238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:12.734354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd830ad1-83d6-4ff2-9edb-7f6cfc309238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:12.735939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd830ad1-83d6-4ff2-9edb-7f6cfc309238 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-bd830ad1-83d6-4ff2-9edb-7f6cfc309238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:32:12.751847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:12.753088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:32:12.754185 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:32:12.754185 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:12.754185 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:12.760176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 458/919] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:12.761356 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237993a-0133-4db9-82ab-23cff94bc414 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:12.782620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:12.883496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237993a-0133-4db9-82ab-23cff94bc414 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:12.927407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1be9e9-0851-425a-80db-a8d6f01f56da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:12.928021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be9e9-0851-425a-80db-a8d6f01f56da None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:12.928297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be9e9-0851-425a-80db-a8d6f01f56da None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:12.928560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1be9e9-0851-425a-80db-a8d6f01f56da None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:12.929188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1be9e9-0851-425a-80db-a8d6f01f56da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:12.929485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1be9e9-0851-425a-80db-a8d6f01f56da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:12.930857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1be9e9-0851-425a-80db-a8d6f01f56da None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:12.943382 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 462/920] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:12.949859 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd830ad1-83d6-4ff2-9edb-7f6cfc309238 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:12.971821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:13.085647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd830ad1-83d6-4ff2-9edb-7f6cfc309238 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:13.127557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b89fe-37ef-42f1-b59f-bc21f8d6f0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:13.128221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b89fe-37ef-42f1-b59f-bc21f8d6f0fa None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:13.128522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b89fe-37ef-42f1-b59f-bc21f8d6f0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:13.128789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b89fe-37ef-42f1-b59f-bc21f8d6f0fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:13.135956 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b89fe-37ef-42f1-b59f-bc21f8d6f0fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:13.137333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b89fe-37ef-42f1-b59f-bc21f8d6f0fa None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-313b89fe-37ef-42f1-b59f-bc21f8d6f0fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:32:13.148934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:13.150000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:13.155975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:32:13.155975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:32:13.155975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:32:13.155975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:32:13.155975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:32:13.155975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:32:13.155975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:32:13.155975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:13.155975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 459/921] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:32:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:13.156479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1be9e9-0851-425a-80db-a8d6f01f56da None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:13.261182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:13.292475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1be9e9-0851-425a-80db-a8d6f01f56da None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:13.331891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903a604-9a2f-4c44-a1e6-bc9351542abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:13.332537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903a604-9a2f-4c44-a1e6-bc9351542abb None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:13.332800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903a604-9a2f-4c44-a1e6-bc9351542abb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:13.333071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903a604-9a2f-4c44-a1e6-bc9351542abb None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:13.333782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903a604-9a2f-4c44-a1e6-bc9351542abb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:13.334087 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903a604-9a2f-4c44-a1e6-bc9351542abb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:13.335595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903a604-9a2f-4c44-a1e6-bc9351542abb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:13.348350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 463/922] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:32:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:13.354330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b89fe-37ef-42f1-b59f-bc21f8d6f0fa None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:13.445968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:13.472223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b89fe-37ef-42f1-b59f-bc21f8d6f0fa None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:13.514268 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab58547-9f01-48fb-b461-b640c620c4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:13.514923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58547-9f01-48fb-b461-b640c620c4c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:13.515197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58547-9f01-48fb-b461-b640c620c4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:13.515464 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab58547-9f01-48fb-b461-b640c620c4c8 None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/e0ce1753e22d4952ab5d4348b0cffc5f/roles/037adbca67514eda861c68218a60c50a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:13.532619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab58547-9f01-48fb-b461-b640c620c4c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=e0ce1753e22d4952ab5d4348b0cffc5f, role_id=037adbca67514eda861c68218a60c50a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:13.534505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab58547-9f01-48fb-b461-b640c620c4c8 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:13.549432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 460/923] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:32:13 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/e0ce1753e22d4952ab5d4348b0cffc5f/roles/037adbca67514eda861c68218a60c50a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:32:15.709237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb2073-b841-40ca-b301-81cd70749485 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:15.709787 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb2073-b841-40ca-b301-81cd70749485 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:15.709787 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb2073-b841-40ca-b301-81cd70749485 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:15.711295 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 464/924] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:32:15.716287 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752bc5f-ff81-41b9-9781-9903a7332218 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:15.716725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752bc5f-ff81-41b9-9781-9903a7332218 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:15.716825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752bc5f-ff81-41b9-9781-9903a7332218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:15.735334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b752bc5f-ff81-41b9-9781-9903a7332218 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:15.736584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b752bc5f-ff81-41b9-9781-9903a7332218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:15.806194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 461/925] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:32:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:32:16.085036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb2073-b841-40ca-b301-81cd70749485 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:16.196033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb2073-b841-40ca-b301-81cd70749485 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:16.234883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb54194-3974-4bb9-b1af-8b9469d10a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:16.235586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb54194-3974-4bb9-b1af-8b9469d10a42 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:16.236054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb54194-3974-4bb9-b1af-8b9469d10a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:16.236416 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb54194-3974-4bb9-b1af-8b9469d10a42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:16.247572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:16.252132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb54194-3974-4bb9-b1af-8b9469d10a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-cdb54194-3974-4bb9-b1af-8b9469d10a42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:32:16.258372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:16.259591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:32:16.260620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 465/926] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Oct 24 13:32:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:16.266054 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752bc5f-ff81-41b9-9781-9903a7332218 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:16.328616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:16.386782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752bc5f-ff81-41b9-9781-9903a7332218 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:16.423019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916e774-99be-4a7d-929f-cfcd58747304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:16.424177 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916e774-99be-4a7d-929f-cfcd58747304 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:16.424454 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916e774-99be-4a7d-929f-cfcd58747304 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:16.424724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916e774-99be-4a7d-929f-cfcd58747304 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:16.425386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916e774-99be-4a7d-929f-cfcd58747304 None admin] RBAC: Adding query filter params (name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:16.425721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916e774-99be-4a7d-929f-cfcd58747304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:16.427129 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916e774-99be-4a7d-929f-cfcd58747304 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:16.435268 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 462/927] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Oct 24 13:32:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:16.442130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb54194-3974-4bb9-b1af-8b9469d10a42 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:16.557507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb54194-3974-4bb9-b1af-8b9469d10a42 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:16.595556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7d9a2-2d1e-4f1c-b727-446eb442febd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:16.596163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7d9a2-2d1e-4f1c-b727-446eb442febd None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:16.596476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7d9a2-2d1e-4f1c-b727-446eb442febd None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:16.596747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7d9a2-2d1e-4f1c-b727-446eb442febd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:16.607902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7d9a2-2d1e-4f1c-b727-446eb442febd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:16.609334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7d9a2-2d1e-4f1c-b727-446eb442febd None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-8fd7d9a2-2d1e-4f1c-b727-446eb442febd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:32:16.628116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:16.629107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:32:16.629989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:32:16.629989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:16.629989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:32:16.629989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 466/928] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:32:16.635542 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916e774-99be-4a7d-929f-cfcd58747304 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:16.747103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916e774-99be-4a7d-929f-cfcd58747304 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:16.769738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:16.789076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda6840-fdb5-4327-af20-317c837a749a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:16.789923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda6840-fdb5-4327-af20-317c837a749a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:16.790224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda6840-fdb5-4327-af20-317c837a749a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:16.790490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda6840-fdb5-4327-af20-317c837a749a None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:16.791167 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda6840-fdb5-4327-af20-317c837a749a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:16.791456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda6840-fdb5-4327-af20-317c837a749a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:16.793002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda6840-fdb5-4327-af20-317c837a749a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:16.805840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 463/929] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:16.811898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7d9a2-2d1e-4f1c-b727-446eb442febd None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:16.869939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:16.927734 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7d9a2-2d1e-4f1c-b727-446eb442febd None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:16.967417 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68892be-b01b-4e3a-bccb-ca857564c773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:16.968237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68892be-b01b-4e3a-bccb-ca857564c773 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:16.968653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68892be-b01b-4e3a-bccb-ca857564c773 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:16.969065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68892be-b01b-4e3a-bccb-ca857564c773 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:16.980094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68892be-b01b-4e3a-bccb-ca857564c773 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:16.981728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68892be-b01b-4e3a-bccb-ca857564c773 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-b68892be-b01b-4e3a-bccb-ca857564c773 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:32:16.994819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:16.995956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:16.996914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:16.996914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:16.996914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:32:16.996914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:32:16.996914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:32:16.996914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:32:16.996914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:32:16.996914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:32:16.996914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:32:16.996914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:32:16.997732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 467/930] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:17.002268 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda6840-fdb5-4327-af20-317c837a749a None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:17.113546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda6840-fdb5-4327-af20-317c837a749a None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:17.149293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8ee54-b3de-443e-b8ea-dc4adec2f181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:17.149900 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8ee54-b3de-443e-b8ea-dc4adec2f181 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:17.150199 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8ee54-b3de-443e-b8ea-dc4adec2f181 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:17.150502 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8ee54-b3de-443e-b8ea-dc4adec2f181 None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:17.151221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8ee54-b3de-443e-b8ea-dc4adec2f181 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:17.151520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8ee54-b3de-443e-b8ea-dc4adec2f181 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:17.152941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8ee54-b3de-443e-b8ea-dc4adec2f181 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:17.171196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 464/931] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Oct 24 13:32:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:17.179861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68892be-b01b-4e3a-bccb-ca857564c773 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:17.295870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68892be-b01b-4e3a-bccb-ca857564c773 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:17.308503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:17.336901 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b7876-b736-43f5-91c6-e0808253160f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:17.337885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b7876-b736-43f5-91c6-e0808253160f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:17.338286 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b7876-b736-43f5-91c6-e0808253160f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:17.338563 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b7876-b736-43f5-91c6-e0808253160f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:17.418788 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b7876-b736-43f5-91c6-e0808253160f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:17.420224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b7876-b736-43f5-91c6-e0808253160f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:17.529271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 468/932] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:32:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:32:17.536585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8ee54-b3de-443e-b8ea-dc4adec2f181 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:17.570683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:17.640648 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8ee54-b3de-443e-b8ea-dc4adec2f181 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:17.679768 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636d699-14e0-4b0a-9e25-0dfc8f126fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:17.680409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636d699-14e0-4b0a-9e25-0dfc8f126fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:17.680688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636d699-14e0-4b0a-9e25-0dfc8f126fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:17.680958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636d699-14e0-4b0a-9e25-0dfc8f126fb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:17.692484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636d699-14e0-4b0a-9e25-0dfc8f126fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:17.694070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636d699-14e0-4b0a-9e25-0dfc8f126fb4 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-b636d699-14e0-4b0a-9e25-0dfc8f126fb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:32:17.710424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:17.711595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:32:17.712618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:32:17.712618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:17.712618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:17.712618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 465/933] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:17.717443 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b7876-b736-43f5-91c6-e0808253160f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:17.729235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:17.824187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b7876-b736-43f5-91c6-e0808253160f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:17.863774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f111578-9606-42e9-b641-db3f0a571796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:17.864499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f111578-9606-42e9-b641-db3f0a571796 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:17.864889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f111578-9606-42e9-b641-db3f0a571796 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:17.865182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f111578-9606-42e9-b641-db3f0a571796 None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:17.865994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f111578-9606-42e9-b641-db3f0a571796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:17.866444 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f111578-9606-42e9-b641-db3f0a571796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:17.868244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f111578-9606-42e9-b641-db3f0a571796 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:17.884090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 469/934] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:17.889818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636d699-14e0-4b0a-9e25-0dfc8f126fb4 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:17.982467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:17.999046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636d699-14e0-4b0a-9e25-0dfc8f126fb4 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:18.036563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9aad24-6c81-4d54-92ec-1250e7b39023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:18.037186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9aad24-6c81-4d54-92ec-1250e7b39023 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:18.037463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9aad24-6c81-4d54-92ec-1250e7b39023 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:18.037738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9aad24-6c81-4d54-92ec-1250e7b39023 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:18.044679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9aad24-6c81-4d54-92ec-1250e7b39023 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:18.046103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9aad24-6c81-4d54-92ec-1250e7b39023 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-dc9aad24-6c81-4d54-92ec-1250e7b39023 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:32:18.058704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:18.061334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:18.062491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:32:18.062491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:32:18.062491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:32:18.062491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:32:18.062491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:32:18.062491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:32:18.062491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:32:18.062491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:18.062491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 466/935] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:32:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:18.066726 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f111578-9606-42e9-b641-db3f0a571796 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:18.140720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:18.175967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f111578-9606-42e9-b641-db3f0a571796 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:18.219577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbde11-c6eb-46e8-945e-39e1abdfc201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:18.220184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbde11-c6eb-46e8-945e-39e1abdfc201 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:18.220651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbde11-c6eb-46e8-945e-39e1abdfc201 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:18.221019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbde11-c6eb-46e8-945e-39e1abdfc201 None admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:18.221663 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbde11-c6eb-46e8-945e-39e1abdfc201 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:18.221957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbde11-c6eb-46e8-945e-39e1abdfc201 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:18.223589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbde11-c6eb-46e8-945e-39e1abdfc201 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:18.236702 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 470/936] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:32:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:18.243129 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9aad24-6c81-4d54-92ec-1250e7b39023 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:18.354937 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9aad24-6c81-4d54-92ec-1250e7b39023 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:18.393287 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f54678-7d4a-4eeb-a740-6b76849c945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:18.393919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f54678-7d4a-4eeb-a740-6b76849c945e None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:18.394228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f54678-7d4a-4eeb-a740-6b76849c945e None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:18.394518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f54678-7d4a-4eeb-a740-6b76849c945e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:18.395337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f54678-7d4a-4eeb-a740-6b76849c945e None admin] RBAC: Adding query filter params (role.id=037adbca67514eda861c68218a60c50a, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=e0ce1753e22d4952ab5d4348b0cffc5f) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:18.395716 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f54678-7d4a-4eeb-a740-6b76849c945e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:18.397292 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f54678-7d4a-4eeb-a740-6b76849c945e None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:18.403293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 467/937] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:32:18 2022] GET /identity/v3/role_assignments?user.id=e0ce1753e22d4952ab5d4348b0cffc5f&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=037adbca67514eda861c68218a60c50a => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:20.778152 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e367bb-adf3-4de6-9d42-56866dcbf59f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:20.778722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e367bb-adf3-4de6-9d42-56866dcbf59f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:20.778722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e367bb-adf3-4de6-9d42-56866dcbf59f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:20.782203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 471/938] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:32:20.785762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377dc1-db09-4929-8619-f3936a8e0157 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:20.786202 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377dc1-db09-4929-8619-f3936a8e0157 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:20.786344 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377dc1-db09-4929-8619-f3936a8e0157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:20.804947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c7377dc1-db09-4929-8619-f3936a8e0157 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:20.806183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c7377dc1-db09-4929-8619-f3936a8e0157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:20.809049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:20.885150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 468/939] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:32:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:32:21.154431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e367bb-adf3-4de6-9d42-56866dcbf59f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:21.267168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e367bb-adf3-4de6-9d42-56866dcbf59f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:21.296876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:21.303639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360fd87-08f2-4889-8233-a3dc11e41d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:21.304298 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360fd87-08f2-4889-8233-a3dc11e41d38 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:21.304580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360fd87-08f2-4889-8233-a3dc11e41d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:21.304850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360fd87-08f2-4889-8233-a3dc11e41d38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:21.316451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360fd87-08f2-4889-8233-a3dc11e41d38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-d360fd87-08f2-4889-8233-a3dc11e41d38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:32:21.321268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:21.322519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:32:21.323744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 472/940] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:32:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:21.328405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377dc1-db09-4929-8619-f3936a8e0157 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:21.435712 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377dc1-db09-4929-8619-f3936a8e0157 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:21.473146 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a95bce-6d63-4e0c-a9fd-f1edc86cbf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:21.473835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a95bce-6d63-4e0c-a9fd-f1edc86cbf06 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:21.474117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a95bce-6d63-4e0c-a9fd-f1edc86cbf06 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:21.474531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a95bce-6d63-4e0c-a9fd-f1edc86cbf06 None admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:21.475187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a95bce-6d63-4e0c-a9fd-f1edc86cbf06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:21.475490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a95bce-6d63-4e0c-a9fd-f1edc86cbf06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:21.477016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a95bce-6d63-4e0c-a9fd-f1edc86cbf06 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:21.486615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 469/941] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:32:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:21.492754 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360fd87-08f2-4889-8233-a3dc11e41d38 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:21.604152 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360fd87-08f2-4889-8233-a3dc11e41d38 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:21.643332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d5e7a-5feb-4763-9203-df8ed5ee7196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:21.644143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d5e7a-5feb-4763-9203-df8ed5ee7196 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:21.644441 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d5e7a-5feb-4763-9203-df8ed5ee7196 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:21.644778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d5e7a-5feb-4763-9203-df8ed5ee7196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:21.655931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d5e7a-5feb-4763-9203-df8ed5ee7196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:21.657401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d5e7a-5feb-4763-9203-df8ed5ee7196 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f48d5e7a-5feb-4763-9203-df8ed5ee7196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:32:21.676345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:21.677448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:32:21.678842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:32:21.678842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:21.678842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:32:21.678842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 473/942] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:21.683360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a95bce-6d63-4e0c-a9fd-f1edc86cbf06 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:21.686414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:21.789035 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a95bce-6d63-4e0c-a9fd-f1edc86cbf06 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:21.826743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97737167-a83e-4b23-86e4-997bf4d8369f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:21.827646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97737167-a83e-4b23-86e4-997bf4d8369f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:21.827931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97737167-a83e-4b23-86e4-997bf4d8369f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:21.828205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97737167-a83e-4b23-86e4-997bf4d8369f None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:21.828889 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97737167-a83e-4b23-86e4-997bf4d8369f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:21.829279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97737167-a83e-4b23-86e4-997bf4d8369f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:21.831625 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97737167-a83e-4b23-86e4-997bf4d8369f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:21.846075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 470/943] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:21.852121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d5e7a-5feb-4763-9203-df8ed5ee7196 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:21.906533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:21.973553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d5e7a-5feb-4763-9203-df8ed5ee7196 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:22.012246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccba935-a434-4b7e-b183-1fb83f940f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:22.012923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccba935-a434-4b7e-b183-1fb83f940f2a None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:22.013207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccba935-a434-4b7e-b183-1fb83f940f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:22.013479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccba935-a434-4b7e-b183-1fb83f940f2a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:22.022706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccba935-a434-4b7e-b183-1fb83f940f2a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:22.024271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccba935-a434-4b7e-b183-1fb83f940f2a None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-3ccba935-a434-4b7e-b183-1fb83f940f2a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:32:22.037361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:22.038700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:22.039953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:22.039953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:22.039953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:32:22.039953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:32:22.039953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:32:22.039953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:32:22.039953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:32:22.039953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:32:22.039953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:32:22.039953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:32:22.039953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 474/944] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:22.044958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97737167-a83e-4b23-86e4-997bf4d8369f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:22.114158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:22.161582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97737167-a83e-4b23-86e4-997bf4d8369f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:22.198789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd46e95-9b41-4d42-a198-c6f6df8f326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:22.199394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd46e95-9b41-4d42-a198-c6f6df8f326c None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:22.199671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd46e95-9b41-4d42-a198-c6f6df8f326c None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:22.199931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd46e95-9b41-4d42-a198-c6f6df8f326c None admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:22.200650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd46e95-9b41-4d42-a198-c6f6df8f326c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:22.200944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd46e95-9b41-4d42-a198-c6f6df8f326c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:22.202419 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd46e95-9b41-4d42-a198-c6f6df8f326c None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:22.221364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 471/945] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Oct 24 13:32:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:22.227410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccba935-a434-4b7e-b183-1fb83f940f2a None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:22.343521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccba935-a434-4b7e-b183-1fb83f940f2a None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:22.345245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:22.385345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317db6d-2753-47a0-b974-63bdbf5a80fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:22.386131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317db6d-2753-47a0-b974-63bdbf5a80fa None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:22.386492 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317db6d-2753-47a0-b974-63bdbf5a80fa None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:22.386800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317db6d-2753-47a0-b974-63bdbf5a80fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:22.399660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317db6d-2753-47a0-b974-63bdbf5a80fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:22.401252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317db6d-2753-47a0-b974-63bdbf5a80fa None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-d317db6d-2753-47a0-b974-63bdbf5a80fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:32:22.417991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:22.419408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:32:22.420519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:32:22.420519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:22.420519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:32:22.420519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 475/946] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:22.426983 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd46e95-9b41-4d42-a198-c6f6df8f326c None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:22.534581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd46e95-9b41-4d42-a198-c6f6df8f326c None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:22.555525 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:22.571649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557505a-30c3-42a2-af3b-ed1d7f09bed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:22.572258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557505a-30c3-42a2-af3b-ed1d7f09bed0 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:22.572535 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557505a-30c3-42a2-af3b-ed1d7f09bed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:22.572803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557505a-30c3-42a2-af3b-ed1d7f09bed0 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:22.573441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557505a-30c3-42a2-af3b-ed1d7f09bed0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:22.573749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557505a-30c3-42a2-af3b-ed1d7f09bed0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:22.575132 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557505a-30c3-42a2-af3b-ed1d7f09bed0 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:22.587293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 472/947] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:22.593474 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317db6d-2753-47a0-b974-63bdbf5a80fa None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:22.713046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317db6d-2753-47a0-b974-63bdbf5a80fa None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:22.756598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3997683-518f-4203-8266-b29836a45126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:22.757448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3997683-518f-4203-8266-b29836a45126 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:22.757843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3997683-518f-4203-8266-b29836a45126 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:22.758348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3997683-518f-4203-8266-b29836a45126 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:22.766691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3997683-518f-4203-8266-b29836a45126 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:22.768675 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3997683-518f-4203-8266-b29836a45126 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-f3997683-518f-4203-8266-b29836a45126 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:32:22.782866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:22.784307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:22.785745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:32:22.785745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:32:22.785745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:32:22.785745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:32:22.785745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:32:22.785745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:32:22.785745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:32:22.785745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:32:22.785745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 476/948] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:32:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:22.790328 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557505a-30c3-42a2-af3b-ed1d7f09bed0 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:22.908855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557505a-30c3-42a2-af3b-ed1d7f09bed0 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:22.946621 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00010d13-0047-4944-aed5-46e364957715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:22.947289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00010d13-0047-4944-aed5-46e364957715 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:22.947591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00010d13-0047-4944-aed5-46e364957715 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:22.947835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00010d13-0047-4944-aed5-46e364957715 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:22.948620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00010d13-0047-4944-aed5-46e364957715 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:22.948921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00010d13-0047-4944-aed5-46e364957715 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:22.950406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00010d13-0047-4944-aed5-46e364957715 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:22.963150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 473/949] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:32:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:22.969152 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3997683-518f-4203-8266-b29836a45126 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:22.987338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:23.082825 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3997683-518f-4203-8266-b29836a45126 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:23.118425 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca17a9e-de22-45a4-b236-539c7353cb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:23.119127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca17a9e-de22-45a4-b236-539c7353cb6b None admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:23.119404 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca17a9e-de22-45a4-b236-539c7353cb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:23.119711 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca17a9e-de22-45a4-b236-539c7353cb6b None admin] PATH_INFO: `/v3/projects/5fd8784d747f495795c413736ac5280c/users/e0ce1753e22d4952ab5d4348b0cffc5f/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:23.121921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca17a9e-de22-45a4-b236-539c7353cb6b None admin] RBAC: Authorizing `identity:create_grant(project_id=5fd8784d747f495795c413736ac5280c, user_id=e0ce1753e22d4952ab5d4348b0cffc5f, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:23.123427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca17a9e-de22-45a4-b236-539c7353cb6b None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:23.136782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 477/950] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Oct 24 13:32:22 2022] PUT /identity/v3/projects/5fd8784d747f495795c413736ac5280c/users/e0ce1753e22d4952ab5d4348b0cffc5f/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:32:25.490212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc74a6-8332-45d8-a5ed-40dbdfaec72d None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:25.490801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc74a6-8332-45d8-a5ed-40dbdfaec72d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:25.490801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc74a6-8332-45d8-a5ed-40dbdfaec72d None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:25.492439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 474/951] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:32:25.498033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381713b-abe2-4d6d-8f80-0fe507e38edf None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:25.498529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381713b-abe2-4d6d-8f80-0fe507e38edf None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:25.498670 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381713b-abe2-4d6d-8f80-0fe507e38edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:25.518093 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a381713b-abe2-4d6d-8f80-0fe507e38edf None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:25.519685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a381713b-abe2-4d6d-8f80-0fe507e38edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:25.593580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 478/952] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:32:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:32:25.874283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc74a6-8332-45d8-a5ed-40dbdfaec72d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:25.924984 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:25.987999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc74a6-8332-45d8-a5ed-40dbdfaec72d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:26.025439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2632f-7b02-45b0-8d9d-d695080ae3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:26.026063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2632f-7b02-45b0-8d9d-d695080ae3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:26.026425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2632f-7b02-45b0-8d9d-d695080ae3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:26.026723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2632f-7b02-45b0-8d9d-d695080ae3d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:26.037451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2632f-7b02-45b0-8d9d-d695080ae3d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-37f2632f-7b02-45b0-8d9d-d695080ae3d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:32:26.042745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:26.043906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:26.044957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 475/953] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Oct 24 13:32:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:26.049988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381713b-abe2-4d6d-8f80-0fe507e38edf None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:26.053088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:26.155910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381713b-abe2-4d6d-8f80-0fe507e38edf None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:26.196760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20e162-1998-4c72-81cf-f9e05720be10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:26.197370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20e162-1998-4c72-81cf-f9e05720be10 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:26.197652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20e162-1998-4c72-81cf-f9e05720be10 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:26.197918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20e162-1998-4c72-81cf-f9e05720be10 None admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:26.198606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20e162-1998-4c72-81cf-f9e05720be10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:26.198945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20e162-1998-4c72-81cf-f9e05720be10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:26.200482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20e162-1998-4c72-81cf-f9e05720be10 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:26.210606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 479/954] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Oct 24 13:32:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:26.216449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2632f-7b02-45b0-8d9d-d695080ae3d1 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:26.325952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:26.328993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2632f-7b02-45b0-8d9d-d695080ae3d1 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:26.366450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f330d0-4734-4ffd-96d7-c764419ad3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:26.367438 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f330d0-4734-4ffd-96d7-c764419ad3bf None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:26.367756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f330d0-4734-4ffd-96d7-c764419ad3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:26.368079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f330d0-4734-4ffd-96d7-c764419ad3bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:26.380188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f330d0-4734-4ffd-96d7-c764419ad3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:26.381879 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f330d0-4734-4ffd-96d7-c764419ad3bf None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-54f330d0-4734-4ffd-96d7-c764419ad3bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:32:26.399955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:26.401135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:32:26.402049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:32:26.402049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:26.402049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:26.402049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 476/955] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:26.407345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20e162-1998-4c72-81cf-f9e05720be10 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:26.479703 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:26.519390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20e162-1998-4c72-81cf-f9e05720be10 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:26.554862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad396153-3c66-4f0b-8d69-ed197ce5529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:26.555483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad396153-3c66-4f0b-8d69-ed197ce5529c None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:26.555794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad396153-3c66-4f0b-8d69-ed197ce5529c None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:26.556008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad396153-3c66-4f0b-8d69-ed197ce5529c None admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:26.556669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad396153-3c66-4f0b-8d69-ed197ce5529c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:26.556945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad396153-3c66-4f0b-8d69-ed197ce5529c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:26.558330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad396153-3c66-4f0b-8d69-ed197ce5529c None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:26.571416 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 480/956] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:32:26.577844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f330d0-4734-4ffd-96d7-c764419ad3bf None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:26.681145 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f330d0-4734-4ffd-96d7-c764419ad3bf None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:26.715695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e73a44-4f92-406c-9ead-6bb7323e7205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:26.716321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e73a44-4f92-406c-9ead-6bb7323e7205 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:26.716604 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e73a44-4f92-406c-9ead-6bb7323e7205 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:26.716879 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e73a44-4f92-406c-9ead-6bb7323e7205 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:26.725361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e73a44-4f92-406c-9ead-6bb7323e7205 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:26.726860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e73a44-4f92-406c-9ead-6bb7323e7205 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-b5e73a44-4f92-406c-9ead-6bb7323e7205 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:32:26.738583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:26.739585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:26.740546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:26.740546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:26.740546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:32:26.740546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:32:26.740546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:32:26.740546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:32:26.740546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:32:26.740546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:32:26.740546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:32:26.740546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:26.740546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 477/957] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:26.747585 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad396153-3c66-4f0b-8d69-ed197ce5529c None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:26.855080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad396153-3c66-4f0b-8d69-ed197ce5529c None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:26.886581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:26.897426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41cbe1-35b0-44bc-8cf3-c590d3ac0a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:26.898046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41cbe1-35b0-44bc-8cf3-c590d3ac0a2f None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:26.898340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41cbe1-35b0-44bc-8cf3-c590d3ac0a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:26.898601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41cbe1-35b0-44bc-8cf3-c590d3ac0a2f None admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:26.899377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41cbe1-35b0-44bc-8cf3-c590d3ac0a2f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:26.899723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41cbe1-35b0-44bc-8cf3-c590d3ac0a2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:26.901137 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41cbe1-35b0-44bc-8cf3-c590d3ac0a2f None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:26.921150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 481/958] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Oct 24 13:32:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:26.927895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e73a44-4f92-406c-9ead-6bb7323e7205 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:26.948346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:27.047529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e73a44-4f92-406c-9ead-6bb7323e7205 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:27.084911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4893338-b94d-4fae-92c8-b0668069a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:27.085582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893338-b94d-4fae-92c8-b0668069a458 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:27.085925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893338-b94d-4fae-92c8-b0668069a458 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:27.086236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893338-b94d-4fae-92c8-b0668069a458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:27.163485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4893338-b94d-4fae-92c8-b0668069a458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:27.165211 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4893338-b94d-4fae-92c8-b0668069a458 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:27.180504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:27.272782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 478/959] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Oct 24 13:32:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:32:27.278690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41cbe1-35b0-44bc-8cf3-c590d3ac0a2f None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:27.385468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41cbe1-35b0-44bc-8cf3-c590d3ac0a2f None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:27.421258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59465d07-9451-4ac1-9fc0-3302f9eb2ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:27.421907 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59465d07-9451-4ac1-9fc0-3302f9eb2ffb None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:27.422205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59465d07-9451-4ac1-9fc0-3302f9eb2ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:27.422481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59465d07-9451-4ac1-9fc0-3302f9eb2ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:27.433660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59465d07-9451-4ac1-9fc0-3302f9eb2ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:27.435157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59465d07-9451-4ac1-9fc0-3302f9eb2ffb None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-59465d07-9451-4ac1-9fc0-3302f9eb2ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:32:27.451954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:27.453253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:32:27.454281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:32:27.454281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:32:27.454281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:32:27.454281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 482/960] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:27.459536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4893338-b94d-4fae-92c8-b0668069a458 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:27.567203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4893338-b94d-4fae-92c8-b0668069a458 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:27.581282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:27.602945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1aec9-a066-4413-a912-5187025dcc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:27.603582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1aec9-a066-4413-a912-5187025dcc22 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:27.603862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1aec9-a066-4413-a912-5187025dcc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:27.604134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1aec9-a066-4413-a912-5187025dcc22 None admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:27.604772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1aec9-a066-4413-a912-5187025dcc22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:27.605074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1aec9-a066-4413-a912-5187025dcc22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:27.606474 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1aec9-a066-4413-a912-5187025dcc22 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:27.618100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 479/961] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Oct 24 13:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:27.623760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59465d07-9451-4ac1-9fc0-3302f9eb2ffb None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:27.662061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:27.729279 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59465d07-9451-4ac1-9fc0-3302f9eb2ffb None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:27.763598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcc316-42bf-479e-ad7f-586d8014b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:27.764232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcc316-42bf-479e-ad7f-586d8014b6ce None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:27.764513 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcc316-42bf-479e-ad7f-586d8014b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:27.764786 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcc316-42bf-479e-ad7f-586d8014b6ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:27.771789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcc316-42bf-479e-ad7f-586d8014b6ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:27.773219 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcc316-42bf-479e-ad7f-586d8014b6ce None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application [None req-99fcc316-42bf-479e-ad7f-586d8014b6ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:32:27.784560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:27.785618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:27.786725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:32:27.786725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:32:27.786725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:32:27.786725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:32:27.786725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:32:27.786725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:32:27.786725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:32:27.786725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: ERROR keystone.server.flask.application Oct 24 13:32:27.786725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 483/962] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Oct 24 13:32:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:27.791872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1aec9-a066-4413-a912-5187025dcc22 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:27.900053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1aec9-a066-4413-a912-5187025dcc22 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:27.932688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cfe289-f592-4da9-9540-3af50f9a6c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:27.933281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfe289-f592-4da9-9540-3af50f9a6c21 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:27.933558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfe289-f592-4da9-9540-3af50f9a6c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:27.933828 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfe289-f592-4da9-9540-3af50f9a6c21 None admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:27.934541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cfe289-f592-4da9-9540-3af50f9a6c21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:27.934844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cfe289-f592-4da9-9540-3af50f9a6c21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:27.936256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cfe289-f592-4da9-9540-3af50f9a6c21 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:27.948277 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 480/963] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Oct 24 13:32:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:27.954409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcc316-42bf-479e-ad7f-586d8014b6ce None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:28.045850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:28.052522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcc316-42bf-479e-ad7f-586d8014b6ce None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:28.084522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20dcbf-93e8-4cde-98fb-3f5dd2c68be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:28.085106 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20dcbf-93e8-4cde-98fb-3f5dd2c68be8 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:28.085379 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20dcbf-93e8-4cde-98fb-3f5dd2c68be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:28.085645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20dcbf-93e8-4cde-98fb-3f5dd2c68be8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:28.086374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20dcbf-93e8-4cde-98fb-3f5dd2c68be8 None admin] RBAC: Adding query filter params (role.id=8b32d66a3cb3483c8ede361ad8637f66, scope.project.id=5fd8784d747f495795c413736ac5280c, user.id=e0ce1753e22d4952ab5d4348b0cffc5f) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:28.086676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20dcbf-93e8-4cde-98fb-3f5dd2c68be8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:28.088075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20dcbf-93e8-4cde-98fb-3f5dd2c68be8 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:28.093536 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 484/964] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Oct 24 13:32:27 2022] GET /identity/v3/role_assignments?user.id=e0ce1753e22d4952ab5d4348b0cffc5f&scope.project.id=5fd8784d747f495795c413736ac5280c&role.id=8b32d66a3cb3483c8ede361ad8637f66 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:30.275824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb16250-f725-49b9-8efd-a523d107b1b7 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:30.276356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb16250-f725-49b9-8efd-a523d107b1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:30.276356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb16250-f725-49b9-8efd-a523d107b1b7 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:30.277673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 481/965] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:32:30.282837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c175-a165-47ff-b5b5-9b8aef1ff051 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:30.283285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c175-a165-47ff-b5b5-9b8aef1ff051 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:30.283405 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c175-a165-47ff-b5b5-9b8aef1ff051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:30.301861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3bd4c175-a165-47ff-b5b5-9b8aef1ff051 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:30.303214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3bd4c175-a165-47ff-b5b5-9b8aef1ff051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:30.367980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 485/966] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:32:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:32:30.637830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb16250-f725-49b9-8efd-a523d107b1b7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:30.668802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:30.738123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb16250-f725-49b9-8efd-a523d107b1b7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:30.770985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e256c8b-f413-4fdb-98b8-c7bb1005f541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:30.771596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e256c8b-f413-4fdb-98b8-c7bb1005f541 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:30.771873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e256c8b-f413-4fdb-98b8-c7bb1005f541 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:30.772139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e256c8b-f413-4fdb-98b8-c7bb1005f541 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:30.772704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e256c8b-f413-4fdb-98b8-c7bb1005f541 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:30.774051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e256c8b-f413-4fdb-98b8-c7bb1005f541 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-1e256c8b-f413-4fdb-98b8-c7bb1005f541 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:30.783315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:30.784590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:30.785645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 482/967] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Oct 24 13:32:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:30.790719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4c175-a165-47ff-b5b5-9b8aef1ff051 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:30.886030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:30.898498 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4c175-a165-47ff-b5b5-9b8aef1ff051 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:30.934929 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32e4d9-bd8f-4284-9418-bdf1c140ffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:30.935535 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32e4d9-bd8f-4284-9418-bdf1c140ffce None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:30.935811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32e4d9-bd8f-4284-9418-bdf1c140ffce None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:30.936077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32e4d9-bd8f-4284-9418-bdf1c140ffce None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:30.936696 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32e4d9-bd8f-4284-9418-bdf1c140ffce None admin] RBAC: Adding query filter params (name=placement) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:30.936996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32e4d9-bd8f-4284-9418-bdf1c140ffce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:30.938375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32e4d9-bd8f-4284-9418-bdf1c140ffce None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:30.943177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 486/968] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Oct 24 13:32:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:30.948883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e256c8b-f413-4fdb-98b8-c7bb1005f541 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:31.039898 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:31.045731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e256c8b-f413-4fdb-98b8-c7bb1005f541 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:31.078178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7633fc-b992-433f-9de6-b52739791114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:31.078786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7633fc-b992-433f-9de6-b52739791114 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:31.079064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7633fc-b992-433f-9de6-b52739791114 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:31.079333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7633fc-b992-433f-9de6-b52739791114 None admin] PATH_INFO: `/v3/services` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:31.080617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7633fc-b992-433f-9de6-b52739791114 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:31.080914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7633fc-b992-433f-9de6-b52739791114 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:31.082286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7633fc-b992-433f-9de6-b52739791114 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:31.086549 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 483/969] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Oct 24 13:32:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:33.209067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead8e4-cab1-454f-a22c-834b08e93130 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:33.209611 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead8e4-cab1-454f-a22c-834b08e93130 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:33.209611 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ead8e4-cab1-454f-a22c-834b08e93130 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:33.210966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 487/970] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:32:33.216224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad743d-962c-4ed3-b7f6-c176a8180b56 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:33.216614 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad743d-962c-4ed3-b7f6-c176a8180b56 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:33.216768 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad743d-962c-4ed3-b7f6-c176a8180b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:33.235038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-5cad743d-962c-4ed3-b7f6-c176a8180b56 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:33.236234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5cad743d-962c-4ed3-b7f6-c176a8180b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:33.299956 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 484/971] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:32:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:32:33.569989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead8e4-cab1-454f-a22c-834b08e93130 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:33.668206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ead8e4-cab1-454f-a22c-834b08e93130 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:33.701866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf0130-57bd-4388-996d-9c09e185ac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:33.702481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf0130-57bd-4388-996d-9c09e185ac22 None admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:33.702760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf0130-57bd-4388-996d-9c09e185ac22 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:33.703029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf0130-57bd-4388-996d-9c09e185ac22 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:33.703668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf0130-57bd-4388-996d-9c09e185ac22 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:33.705045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf0130-57bd-4388-996d-9c09e185ac22 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:33.718922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 488/972] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Oct 24 13:32:33 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:32:35.838376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b9e93-b96a-456f-ac68-4f47455e4870 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:35.838957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b9e93-b96a-456f-ac68-4f47455e4870 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:35.838957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b9e93-b96a-456f-ac68-4f47455e4870 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:35.840268 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 485/973] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:32:35.846406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1dedc-60fc-44dc-af76-11dd1faec04d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:35.846874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1dedc-60fc-44dc-af76-11dd1faec04d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:35.846994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1dedc-60fc-44dc-af76-11dd1faec04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:35.867948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3ce1dedc-60fc-44dc-af76-11dd1faec04d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:35.869196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3ce1dedc-60fc-44dc-af76-11dd1faec04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:35.879421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:35.950993 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 489/974] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:32:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:32:36.215518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b9e93-b96a-456f-ac68-4f47455e4870 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:36.304411 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:36.315121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b9e93-b96a-456f-ac68-4f47455e4870 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:36.347754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71549502-a234-4c08-8ac4-d00f44a4b830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:36.348363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549502-a234-4c08-8ac4-d00f44a4b830 None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:36.348640 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549502-a234-4c08-8ac4-d00f44a4b830 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:36.348904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71549502-a234-4c08-8ac4-d00f44a4b830 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:36.349475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71549502-a234-4c08-8ac4-d00f44a4b830 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:36.350853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71549502-a234-4c08-8ac4-d00f44a4b830 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-71549502-a234-4c08-8ac4-d00f44a4b830 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:36.359356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:36.360543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:36.361501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 486/975] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Oct 24 13:32:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:36.366662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1dedc-60fc-44dc-af76-11dd1faec04d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:36.464068 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1dedc-60fc-44dc-af76-11dd1faec04d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:36.496461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1c280-4208-449b-acaf-cfadc3aceea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:36.497058 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1c280-4208-449b-acaf-cfadc3aceea6 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:36.497338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1c280-4208-449b-acaf-cfadc3aceea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:36.497608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1c280-4208-449b-acaf-cfadc3aceea6 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:36.498268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1c280-4208-449b-acaf-cfadc3aceea6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:36.498571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1c280-4208-449b-acaf-cfadc3aceea6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:36.499921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1c280-4208-449b-acaf-cfadc3aceea6 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:36.505381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 490/976] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Oct 24 13:32:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:36.511336 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71549502-a234-4c08-8ac4-d00f44a4b830 None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:36.608340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71549502-a234-4c08-8ac4-d00f44a4b830 None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:36.640314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5b820-9701-4756-a6a1-6b45ba86af3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:36.640899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5b820-9701-4756-a6a1-6b45ba86af3a None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:36.641175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5b820-9701-4756-a6a1-6b45ba86af3a None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:36.641443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5b820-9701-4756-a6a1-6b45ba86af3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:36.642695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5b820-9701-4756-a6a1-6b45ba86af3a None admin] RBAC: Adding query filter params (interface=public, service_id=938a2b46888347d6a27df5b179a062f5, region_id=RegionOne) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:36.642994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5b820-9701-4756-a6a1-6b45ba86af3a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:36.644341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5b820-9701-4756-a6a1-6b45ba86af3a None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:36.649130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 487/977] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Oct 24 13:32:36 2022] GET /identity/v3/endpoints?service_id=938a2b46888347d6a27df5b179a062f5&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:36.654650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1c280-4208-449b-acaf-cfadc3aceea6 None admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:36.658473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:36.750850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1c280-4208-449b-acaf-cfadc3aceea6 None admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:36.784609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a34b0-0cf0-4f2a-8195-6a4ac95362e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:36.785183 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a34b0-0cf0-4f2a-8195-6a4ac95362e3 None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:36.785454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a34b0-0cf0-4f2a-8195-6a4ac95362e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:36.785717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a34b0-0cf0-4f2a-8195-6a4ac95362e3 None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:36.786328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a34b0-0cf0-4f2a-8195-6a4ac95362e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:36.787669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a34b0-0cf0-4f2a-8195-6a4ac95362e3 None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:36.794534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 491/978] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Oct 24 13:32:36 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:32:38.950521 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c1f63-d8a5-485b-922a-931a1a9da481 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:38.951050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c1f63-d8a5-485b-922a-931a1a9da481 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:38.951050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c1f63-d8a5-485b-922a-931a1a9da481 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:38.952384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 488/979] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:32:38.957763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e385-b4a5-404a-a400-d5f68495afe5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:38.958188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e385-b4a5-404a-a400-d5f68495afe5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:38.958343 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2e385-b4a5-404a-a400-d5f68495afe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:38.975953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-cec2e385-b4a5-404a-a400-d5f68495afe5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:38.977111 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cec2e385-b4a5-404a-a400-d5f68495afe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:39.036851 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:39.042755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 492/980] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:32:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:32:39.317519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c1f63-d8a5-485b-922a-931a1a9da481 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:39.337104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:39.420199 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c1f63-d8a5-485b-922a-931a1a9da481 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:39.452911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba22219-c606-422d-8972-1a76c5424d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:39.453524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba22219-c606-422d-8972-1a76c5424d4f None admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:39.453810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba22219-c606-422d-8972-1a76c5424d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:39.454079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba22219-c606-422d-8972-1a76c5424d4f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:39.454867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba22219-c606-422d-8972-1a76c5424d4f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:39.456227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba22219-c606-422d-8972-1a76c5424d4f None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application [None req-4ba22219-c606-422d-8972-1a76c5424d4f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:32:39.464808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:32:39.465934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application with Lock( Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: ERROR keystone.server.flask.application Oct 24 13:32:39.466963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 489/981] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Oct 24 13:32:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:32:39.473642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2e385-b4a5-404a-a400-d5f68495afe5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:39.570323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2e385-b4a5-404a-a400-d5f68495afe5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:39.603949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ed150-aa42-42eb-b230-49f81ed5fe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:39.604558 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed150-aa42-42eb-b230-49f81ed5fe9e None admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:39.604833 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed150-aa42-42eb-b230-49f81ed5fe9e None admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:39.605100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed150-aa42-42eb-b230-49f81ed5fe9e None admin] PATH_INFO: `/v3/services` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:39.605723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ed150-aa42-42eb-b230-49f81ed5fe9e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:39.606023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ed150-aa42-42eb-b230-49f81ed5fe9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:39.607412 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ed150-aa42-42eb-b230-49f81ed5fe9e None admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:39.613347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 493/982] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Oct 24 13:32:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:32:39.619183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba22219-c606-422d-8972-1a76c5424d4f None admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:39.698810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:39.716851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba22219-c606-422d-8972-1a76c5424d4f None admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:39.749596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149f9be-a2c3-4edf-9eb2-4772563bc312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:39.750196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149f9be-a2c3-4edf-9eb2-4772563bc312 None admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:39.750476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149f9be-a2c3-4edf-9eb2-4772563bc312 None admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:39.750743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149f9be-a2c3-4edf-9eb2-4772563bc312 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:39.751723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149f9be-a2c3-4edf-9eb2-4772563bc312 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:39.753158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149f9be-a2c3-4edf-9eb2-4772563bc312 None admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:39.772274 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 490/983] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:32:39 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:32:49.609490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=69 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:49.767815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:43.916609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99209813-6591-4c3e-94c6-0c536e36b111 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:43.917096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99209813-6591-4c3e-94c6-0c536e36b111 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:43.917183 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99209813-6591-4c3e-94c6-0c536e36b111 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:43.924463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98376c-9006-4c4f-a712-04c13937fa91 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:43.924824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98376c-9006-4c4f-a712-04c13937fa91 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:43.924915 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98376c-9006-4c4f-a712-04c13937fa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:43.929715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 494/984] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:33:43 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:33:44.114982 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3e98376c-9006-4c4f-a712-04c13937fa91 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:44.116533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3e98376c-9006-4c4f-a712-04c13937fa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:44.270921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 491/985] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:33:44.573994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46889869-83f6-4ffb-ad76-135ee08b42b8 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:44.574351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46889869-83f6-4ffb-ad76-135ee08b42b8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:44.574553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46889869-83f6-4ffb-ad76-135ee08b42b8 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:44.575908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 495/986] 10.222.0.20 () {54 vars in 964 bytes} [Mon Oct 24 13:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:33:44.582611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dca07-ec72-409c-a2d1-642853925d71 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:44.583066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dca07-ec72-409c-a2d1-642853925d71 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:44.583433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dca07-ec72-409c-a2d1-642853925d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:44.688194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e44dca07-ec72-409c-a2d1-642853925d71 None None] MFA Rules not processed for user `5b8479c571ec449fbfc306d25fbf7fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:44.689644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e44dca07-ec72-409c-a2d1-642853925d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:44.715107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:44.782456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 492/987] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Oct 24 13:33:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:33:44.791456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46889869-83f6-4ffb-ad76-135ee08b42b8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:44.793515 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-46889869-83f6-4ffb-ad76-135ee08b42b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:44.911168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46889869-83f6-4ffb-ad76-135ee08b42b8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:44.991136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c5d84-2b1b-4ba0-9a38-2e2ff5ab83ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:44.991766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c5d84-2b1b-4ba0-9a38-2e2ff5ab83ca service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:44.992037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c5d84-2b1b-4ba0-9a38-2e2ff5ab83ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:44.992234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c5d84-2b1b-4ba0-9a38-2e2ff5ab83ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:44.993834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f93c5d84-2b1b-4ba0-9a38-2e2ff5ab83ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:45.052934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:45.116420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c5d84-2b1b-4ba0-9a38-2e2ff5ab83ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:45.117851 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c5d84-2b1b-4ba0-9a38-2e2ff5ab83ca service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:45.139000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 496/988] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:33:47.745216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0fd87-98ad-4a80-873c-dafa80207e4a None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:47.745767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0fd87-98ad-4a80-873c-dafa80207e4a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:47.745767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0fd87-98ad-4a80-873c-dafa80207e4a None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:47.747172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 493/989] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:33:47.752462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d160cba-2b96-4de4-97cc-b3c616ddb1e4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:47.752886 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d160cba-2b96-4de4-97cc-b3c616ddb1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:47.753182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d160cba-2b96-4de4-97cc-b3c616ddb1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:47.827696 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4d160cba-2b96-4de4-97cc-b3c616ddb1e4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:47.828954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4d160cba-2b96-4de4-97cc-b3c616ddb1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:47.926376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 497/990] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:33:48.207873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0fd87-98ad-4a80-873c-dafa80207e4a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:48.209406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-60f0fd87-98ad-4a80-873c-dafa80207e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:48.312747 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0fd87-98ad-4a80-873c-dafa80207e4a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:48.383317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:48.385664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b831b3-e969-4b0d-ae47-a0ff9eaa9fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:48.386331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b831b3-e969-4b0d-ae47-a0ff9eaa9fea service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:48.386642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b831b3-e969-4b0d-ae47-a0ff9eaa9fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:48.387096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b831b3-e969-4b0d-ae47-a0ff9eaa9fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:48.389100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b6b831b3-e969-4b0d-ae47-a0ff9eaa9fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:48.495556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b831b3-e969-4b0d-ae47-a0ff9eaa9fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:48.496895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b831b3-e969-4b0d-ae47-a0ff9eaa9fea service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:48.514728 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 494/991] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:33:50.988707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420d193-0f75-448f-b526-2315ab1654e5 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:50.989271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420d193-0f75-448f-b526-2315ab1654e5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:50.989271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420d193-0f75-448f-b526-2315ab1654e5 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:50.990663 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 498/992] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:33:50.995941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00425a85-55cc-47a3-b48c-b0a5fb920137 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:50.996337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00425a85-55cc-47a3-b48c-b0a5fb920137 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:50.996476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00425a85-55cc-47a3-b48c-b0a5fb920137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:51.065134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-00425a85-55cc-47a3-b48c-b0a5fb920137 None None] MFA Rules not processed for user `bca379d1a4aa4b3884c2c194379118dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:51.066434 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-00425a85-55cc-47a3-b48c-b0a5fb920137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:51.116691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:51.160735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 495/993] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:33:50 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:33:51.451438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420d193-0f75-448f-b526-2315ab1654e5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:51.453088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0420d193-0f75-448f-b526-2315ab1654e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:51.462801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:51.556008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420d193-0f75-448f-b526-2315ab1654e5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:51.628094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0314e-0d7e-46e0-9599-d8c617583745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:51.628686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0314e-0d7e-46e0-9599-d8c617583745 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:51.628958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0314e-0d7e-46e0-9599-d8c617583745 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:51.629221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0314e-0d7e-46e0-9599-d8c617583745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:51.630804 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-36c0314e-0d7e-46e0-9599-d8c617583745 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:51.691900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:51.738584 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0314e-0d7e-46e0-9599-d8c617583745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:51.739941 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0314e-0d7e-46e0-9599-d8c617583745 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:51.758913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 499/994] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:33:51 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:33:54.431446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205d04f-c7c8-44ce-a7f6-fa8dc4ecc58f None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:54.431994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205d04f-c7c8-44ce-a7f6-fa8dc4ecc58f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:54.431994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205d04f-c7c8-44ce-a7f6-fa8dc4ecc58f None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:54.433332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 496/995] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:33:54.438732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505684de-026d-4ffe-b63c-04674a8cee71 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:54.439127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505684de-026d-4ffe-b63c-04674a8cee71 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:54.439275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505684de-026d-4ffe-b63c-04674a8cee71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:54.503825 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-505684de-026d-4ffe-b63c-04674a8cee71 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:54.504949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-505684de-026d-4ffe-b63c-04674a8cee71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:54.598559 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 500/996] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:33:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:33:54.909906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921390c9-259c-4118-95fb-1ed599680269 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:54.910213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921390c9-259c-4118-95fb-1ed599680269 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:54.910483 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921390c9-259c-4118-95fb-1ed599680269 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:54.911820 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 497/997] 10.222.0.20 () {54 vars in 964 bytes} [Mon Oct 24 13:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:33:54.918347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38efa5f-aeed-405b-95c0-e58c3929220a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:54.918754 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38efa5f-aeed-405b-95c0-e58c3929220a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:54.919034 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38efa5f-aeed-405b-95c0-e58c3929220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:54.940879 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:54.994453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c38efa5f-aeed-405b-95c0-e58c3929220a None None] MFA Rules not processed for user `5b8479c571ec449fbfc306d25fbf7fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:54.995628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c38efa5f-aeed-405b-95c0-e58c3929220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:55.071378 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 501/998] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Oct 24 13:33:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:33:55.081048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921390c9-259c-4118-95fb-1ed599680269 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:55.082617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-921390c9-259c-4118-95fb-1ed599680269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:55.182615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921390c9-259c-4118-95fb-1ed599680269 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:55.257940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a00fbf-062f-47f1-8452-26313669d256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:55.258574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a00fbf-062f-47f1-8452-26313669d256 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:55.258848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a00fbf-062f-47f1-8452-26313669d256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:55.259111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a00fbf-062f-47f1-8452-26313669d256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:55.260672 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-35a00fbf-062f-47f1-8452-26313669d256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:55.267933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:55.371275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a00fbf-062f-47f1-8452-26313669d256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:55.372691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a00fbf-062f-47f1-8452-26313669d256 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:55.390862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 498/999] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:33:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:33:58.767858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e02e3c-5628-4232-b99b-9201ebe1c39c None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:58.768471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e02e3c-5628-4232-b99b-9201ebe1c39c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:58.768471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e02e3c-5628-4232-b99b-9201ebe1c39c None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:58.769945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 502/1000] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:33:58.776905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d45c5-be64-4fff-8b08-2a629ae44f57 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:58.777391 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d45c5-be64-4fff-8b08-2a629ae44f57 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:58.777480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d45c5-be64-4fff-8b08-2a629ae44f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:58.865235 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2e1d45c5-be64-4fff-8b08-2a629ae44f57 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:58.866834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2e1d45c5-be64-4fff-8b08-2a629ae44f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:58.939618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:58.985371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 499/1001] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:33:59.299940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e02e3c-5628-4232-b99b-9201ebe1c39c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:59.301520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-93e02e3c-5628-4232-b99b-9201ebe1c39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:59.408260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e02e3c-5628-4232-b99b-9201ebe1c39c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:59.420494 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:59.484877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d73e7-00ce-4675-85f0-251a069930a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:59.485477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d73e7-00ce-4675-85f0-251a069930a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:59.485746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d73e7-00ce-4675-85f0-251a069930a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:59.486005 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d73e7-00ce-4675-85f0-251a069930a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:59.487619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2b1d73e7-00ce-4675-85f0-251a069930a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:59.594038 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d73e7-00ce-4675-85f0-251a069930a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:59.595473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d73e7-00ce-4675-85f0-251a069930a1 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:59.613975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 503/1002] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:01.928573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0febfe9-e0b3-4ab9-a55d-b17631484240 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:01.929221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0febfe9-e0b3-4ab9-a55d-b17631484240 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:01.929221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0febfe9-e0b3-4ab9-a55d-b17631484240 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:01.930473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 500/1003] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:01.935705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc3e81-3a2d-41ab-9bea-41c412218402 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:01.936108 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc3e81-3a2d-41ab-9bea-41c412218402 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:01.936244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc3e81-3a2d-41ab-9bea-41c412218402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:01.984176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:02.011389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-48dc3e81-3a2d-41ab-9bea-41c412218402 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:02.012586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-48dc3e81-3a2d-41ab-9bea-41c412218402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:02.112417 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 504/1004] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:02.399549 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0febfe9-e0b3-4ab9-a55d-b17631484240 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:02.401142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d0febfe9-e0b3-4ab9-a55d-b17631484240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:02.503753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0febfe9-e0b3-4ab9-a55d-b17631484240 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:02.579197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f76ca-0965-4619-b1ff-bbd2a04ad86e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:02.579800 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f76ca-0965-4619-b1ff-bbd2a04ad86e service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:02.580073 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f76ca-0965-4619-b1ff-bbd2a04ad86e service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:02.580336 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f76ca-0965-4619-b1ff-bbd2a04ad86e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:02.581905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c87f76ca-0965-4619-b1ff-bbd2a04ad86e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:02.599935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:02.692725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f76ca-0965-4619-b1ff-bbd2a04ad86e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:02.694153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f76ca-0965-4619-b1ff-bbd2a04ad86e service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:02.712645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 501/1005] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:04.965340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c869d26-0e6d-4120-975a-06e24de8da31 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:04.965863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c869d26-0e6d-4120-975a-06e24de8da31 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:04.965863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c869d26-0e6d-4120-975a-06e24de8da31 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:04.967161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 505/1006] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:04.972556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5342a-a00d-4593-8f21-24d28af48db0 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:04.972936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5342a-a00d-4593-8f21-24d28af48db0 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:04.973097 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5342a-a00d-4593-8f21-24d28af48db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:05.041918 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7ff5342a-a00d-4593-8f21-24d28af48db0 None None] MFA Rules not processed for user `bca379d1a4aa4b3884c2c194379118dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:05.043150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7ff5342a-a00d-4593-8f21-24d28af48db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:05.107669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:05.139677 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 502/1007] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:05.424011 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c869d26-0e6d-4120-975a-06e24de8da31 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:05.425540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9c869d26-0e6d-4120-975a-06e24de8da31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:05.530580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c869d26-0e6d-4120-975a-06e24de8da31 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:05.551571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:05.608017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a62dc9-ea79-4df5-ad47-8d067baa9df4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:05.609250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62dc9-ea79-4df5-ad47-8d067baa9df4 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:05.609531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62dc9-ea79-4df5-ad47-8d067baa9df4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:05.609798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62dc9-ea79-4df5-ad47-8d067baa9df4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:05.611411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f2a62dc9-ea79-4df5-ad47-8d067baa9df4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:05.724233 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62dc9-ea79-4df5-ad47-8d067baa9df4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:05.725651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62dc9-ea79-4df5-ad47-8d067baa9df4 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:05.744482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 506/1008] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:08.177214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2525-b222-4954-8ec5-0297bcdef8bc None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:08.177757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2525-b222-4954-8ec5-0297bcdef8bc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:08.177757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2525-b222-4954-8ec5-0297bcdef8bc None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:08.179037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 503/1009] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:08.184178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da27c7-f162-4b17-9c5d-16b145cd0e7b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:08.184609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da27c7-f162-4b17-9c5d-16b145cd0e7b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:08.184733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da27c7-f162-4b17-9c5d-16b145cd0e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:08.247996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:08.250992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b4da27c7-f162-4b17-9c5d-16b145cd0e7b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:08.252818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b4da27c7-f162-4b17-9c5d-16b145cd0e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:08.347876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 507/1010] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:08.638492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2525-b222-4954-8ec5-0297bcdef8bc None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:08.640784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-272c2525-b222-4954-8ec5-0297bcdef8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:08.741616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2525-b222-4954-8ec5-0297bcdef8bc None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:08.814719 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f632b-fa64-40f2-84ab-30241f28df10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:08.815312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f632b-fa64-40f2-84ab-30241f28df10 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:08.815588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f632b-fa64-40f2-84ab-30241f28df10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:08.815856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f632b-fa64-40f2-84ab-30241f28df10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:08.817433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-319f632b-fa64-40f2-84ab-30241f28df10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:08.843961 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:08.926260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f632b-fa64-40f2-84ab-30241f28df10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:08.927614 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f632b-fa64-40f2-84ab-30241f28df10 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:08.945731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 504/1011] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:11.147746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283a000-26cf-48b2-8780-054587beede4 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:11.148299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283a000-26cf-48b2-8780-054587beede4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:11.148299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283a000-26cf-48b2-8780-054587beede4 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:11.149605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 508/1012] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:11.154789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5d7b4-238c-4b73-a2c1-a8e89f15c190 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:11.155223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5d7b4-238c-4b73-a2c1-a8e89f15c190 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:11.155333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5d7b4-238c-4b73-a2c1-a8e89f15c190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:11.222134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-dff5d7b4-238c-4b73-a2c1-a8e89f15c190 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:11.223379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-dff5d7b4-238c-4b73-a2c1-a8e89f15c190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:11.297848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:11.319983 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 505/1013] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:11.606007 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283a000-26cf-48b2-8780-054587beede4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:11.607569 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c283a000-26cf-48b2-8780-054587beede4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:11.710358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283a000-26cf-48b2-8780-054587beede4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:11.739248 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:11.783600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96640b77-d81f-4318-b6e7-34a2d7e28735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:11.784198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640b77-d81f-4318-b6e7-34a2d7e28735 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:11.784472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640b77-d81f-4318-b6e7-34a2d7e28735 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:11.784738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640b77-d81f-4318-b6e7-34a2d7e28735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:11.786334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-96640b77-d81f-4318-b6e7-34a2d7e28735 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:11.894397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96640b77-d81f-4318-b6e7-34a2d7e28735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:11.895745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96640b77-d81f-4318-b6e7-34a2d7e28735 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:11.913606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 509/1014] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:14.456580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b727a0-bfe7-4275-b5bb-1f8655309bfa None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:14.457106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b727a0-bfe7-4275-b5bb-1f8655309bfa None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:14.457106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b727a0-bfe7-4275-b5bb-1f8655309bfa None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:14.458430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 506/1015] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:14.464239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80981f-8b09-4adc-890c-572190213f2c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:14.464658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80981f-8b09-4adc-890c-572190213f2c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:14.464760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80981f-8b09-4adc-890c-572190213f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:14.528780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4b80981f-8b09-4adc-890c-572190213f2c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:14.529944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4b80981f-8b09-4adc-890c-572190213f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:14.538110 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:14.624692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 510/1016] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:14.913641 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b727a0-bfe7-4275-b5bb-1f8655309bfa None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:14.915447 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-74b727a0-bfe7-4275-b5bb-1f8655309bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:15.018354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b727a0-bfe7-4275-b5bb-1f8655309bfa None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:15.090423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb541a-8779-492f-88f8-21a8314d97c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:15.091016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb541a-8779-492f-88f8-21a8314d97c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:15.091290 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb541a-8779-492f-88f8-21a8314d97c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:15.091554 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb541a-8779-492f-88f8-21a8314d97c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:15.093965 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-aadb541a-8779-492f-88f8-21a8314d97c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:15.128135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:15.201092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb541a-8779-492f-88f8-21a8314d97c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:15.203309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb541a-8779-492f-88f8-21a8314d97c1 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:15.222319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 507/1017] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:19.391483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecf54a-52bd-467c-9d5a-bd7fd8af024b None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:19.392008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecf54a-52bd-467c-9d5a-bd7fd8af024b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:19.392008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecf54a-52bd-467c-9d5a-bd7fd8af024b None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:19.393366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 511/1018] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:19.398694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f984-2fc8-402e-9e18-987fad129178 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:19.399101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f984-2fc8-402e-9e18-987fad129178 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:19.399272 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63f984-2fc8-402e-9e18-987fad129178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:19.466453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0d63f984-2fc8-402e-9e18-987fad129178 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:19.467541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0d63f984-2fc8-402e-9e18-987fad129178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:19.557061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:19.569462 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 508/1019] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:19.855501 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecf54a-52bd-467c-9d5a-bd7fd8af024b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:19.857233 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5aecf54a-52bd-467c-9d5a-bd7fd8af024b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:19.961455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecf54a-52bd-467c-9d5a-bd7fd8af024b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:19.999241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:20.035770 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47718b-dc06-47ef-8d22-d240c11ec9c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:20.036374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47718b-dc06-47ef-8d22-d240c11ec9c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:20.036655 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47718b-dc06-47ef-8d22-d240c11ec9c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:20.036920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47718b-dc06-47ef-8d22-d240c11ec9c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:20.038506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cb47718b-dc06-47ef-8d22-d240c11ec9c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:20.148052 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47718b-dc06-47ef-8d22-d240c11ec9c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:20.149395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47718b-dc06-47ef-8d22-d240c11ec9c3 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:20.168556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 512/1020] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:22.995955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693d071-267e-4775-a818-b1ca27d9a7d1 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:22.996462 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693d071-267e-4775-a818-b1ca27d9a7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:22.996462 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693d071-267e-4775-a818-b1ca27d9a7d1 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:22.997711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 509/1021] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:34:23.003804 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1cc5c-a1fb-4376-ba18-5f8e92d53494 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:23.004219 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1cc5c-a1fb-4376-ba18-5f8e92d53494 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:23.004320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1cc5c-a1fb-4376-ba18-5f8e92d53494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:23.070922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-63a1cc5c-a1fb-4376-ba18-5f8e92d53494 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:23.072101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-63a1cc5c-a1fb-4376-ba18-5f8e92d53494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:23.090997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:23.170318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 513/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:23.454637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693d071-267e-4775-a818-b1ca27d9a7d1 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:23.456167 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9693d071-267e-4775-a818-b1ca27d9a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:23.565689 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693d071-267e-4775-a818-b1ca27d9a7d1 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:23.639717 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727628e0-26cc-4b47-a382-7f43206d4347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:23.640324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727628e0-26cc-4b47-a382-7f43206d4347 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:23.640603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727628e0-26cc-4b47-a382-7f43206d4347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:23.640873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727628e0-26cc-4b47-a382-7f43206d4347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:23.642524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-727628e0-26cc-4b47-a382-7f43206d4347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:23.687076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:23.755457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727628e0-26cc-4b47-a382-7f43206d4347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:23.756811 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727628e0-26cc-4b47-a382-7f43206d4347 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:23.775916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 510/1023] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:27.997779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577386b-3020-448f-8760-c8eb859d2da2 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:27.997779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577386b-3020-448f-8760-c8eb859d2da2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:27.998395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577386b-3020-448f-8760-c8eb859d2da2 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:28.004194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 514/1024] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:28.005844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cad18-836a-4d5e-993e-1c272b580571 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:28.006329 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cad18-836a-4d5e-993e-1c272b580571 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:28.006495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cad18-836a-4d5e-993e-1c272b580571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:28.079091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a51cad18-836a-4d5e-993e-1c272b580571 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:28.080299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a51cad18-836a-4d5e-993e-1c272b580571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:28.190513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:28.194101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 511/1025] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:28.479221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577386b-3020-448f-8760-c8eb859d2da2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:28.480740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c577386b-3020-448f-8760-c8eb859d2da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:28.584338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577386b-3020-448f-8760-c8eb859d2da2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:28.631102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:28.657911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94461e45-a285-49e4-81ed-096c6425e40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:28.658537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94461e45-a285-49e4-81ed-096c6425e40e service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:28.658813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94461e45-a285-49e4-81ed-096c6425e40e service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:28.659081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94461e45-a285-49e4-81ed-096c6425e40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:28.660644 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-94461e45-a285-49e4-81ed-096c6425e40e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:28.769469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94461e45-a285-49e4-81ed-096c6425e40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:28.770828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94461e45-a285-49e4-81ed-096c6425e40e service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:28.789396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 515/1026] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:33.756112 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadddf1-5e5f-4e6a-9149-24188707ce51 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:33.756590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadddf1-5e5f-4e6a-9149-24188707ce51 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:33.756692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadddf1-5e5f-4e6a-9149-24188707ce51 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:33.758222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 512/1027] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:33.763366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2751aff-94d5-4c94-9c7b-681682a7f112 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:33.763799 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2751aff-94d5-4c94-9c7b-681682a7f112 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:33.763984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2751aff-94d5-4c94-9c7b-681682a7f112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:33.876571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a2751aff-94d5-4c94-9c7b-681682a7f112 None None] MFA Rules not processed for user `e0ce1753e22d4952ab5d4348b0cffc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:33.877881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a2751aff-94d5-4c94-9c7b-681682a7f112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:33.942802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:34.049699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 516/1028] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:34:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:38.142733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ae2e-1dd0-47ff-9b88-ccebd0aed765 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:38.143237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ae2e-1dd0-47ff-9b88-ccebd0aed765 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:38.143237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376ae2e-1dd0-47ff-9b88-ccebd0aed765 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:38.144688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 513/1029] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:38.149485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839d758-8237-4a8f-809a-1fe76f24a48f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:38.150036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839d758-8237-4a8f-809a-1fe76f24a48f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:38.150260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839d758-8237-4a8f-809a-1fe76f24a48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:38.239297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5839d758-8237-4a8f-809a-1fe76f24a48f None None] MFA Rules not processed for user `e0ce1753e22d4952ab5d4348b0cffc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:38.240589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5839d758-8237-4a8f-809a-1fe76f24a48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:38.315646 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:38.353733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 517/1030] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:39.925115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e05281-433e-4fa0-a6d8-877b806c7a83 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:39.925638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e05281-433e-4fa0-a6d8-877b806c7a83 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:39.925638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e05281-433e-4fa0-a6d8-877b806c7a83 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:39.926984 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 514/1031] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:39.932001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca9396-08b3-4bf7-9652-1dd4ad730ea7 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:39.932404 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca9396-08b3-4bf7-9652-1dd4ad730ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:39.932543 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca9396-08b3-4bf7-9652-1dd4ad730ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:40.013818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3dca9396-08b3-4bf7-9652-1dd4ad730ea7 None None] MFA Rules not processed for user `e0ce1753e22d4952ab5d4348b0cffc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:40.015351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3dca9396-08b3-4bf7-9652-1dd4ad730ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:40.129628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 518/1032] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:34:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:43.532660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed08fd-910b-4f90-aa40-db537b8fccd5 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:43.533158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed08fd-910b-4f90-aa40-db537b8fccd5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:43.533158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed08fd-910b-4f90-aa40-db537b8fccd5 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:43.534620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 515/1033] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:43.540358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a79da-75d0-4242-a04d-8dfd34b231a6 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:43.540764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a79da-75d0-4242-a04d-8dfd34b231a6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:43.540905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a79da-75d0-4242-a04d-8dfd34b231a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:43.588249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:43.621714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ab6a79da-75d0-4242-a04d-8dfd34b231a6 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:43.622841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ab6a79da-75d0-4242-a04d-8dfd34b231a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:43.725187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 519/1034] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:44.039174 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8275c9-76c0-4acd-8343-d935ab82bf02 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:44.039459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8275c9-76c0-4acd-8343-d935ab82bf02 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:44.039734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8275c9-76c0-4acd-8343-d935ab82bf02 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:44.041060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 516/1035] 10.222.0.20 () {54 vars in 961 bytes} [Mon Oct 24 13:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:44.049262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcd096-24fe-419b-a07b-7b922ccfbf8a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:44.049780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcd096-24fe-419b-a07b-7b922ccfbf8a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:44.050279 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcd096-24fe-419b-a07b-7b922ccfbf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:44.148871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7edcd096-24fe-419b-a07b-7b922ccfbf8a None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:44.150692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7edcd096-24fe-419b-a07b-7b922ccfbf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:44.224035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:44.304034 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 520/1036] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Oct 24 13:34:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:44.312042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8275c9-76c0-4acd-8343-d935ab82bf02 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:44.313573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9d8275c9-76c0-4acd-8343-d935ab82bf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:44.471220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8275c9-76c0-4acd-8343-d935ab82bf02 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:44.551933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:44.595029 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caa9e5-19a9-45eb-be5e-b292b5646f8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:44.595629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caa9e5-19a9-45eb-be5e-b292b5646f8f service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:44.595907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caa9e5-19a9-45eb-be5e-b292b5646f8f service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:44.596172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caa9e5-19a9-45eb-be5e-b292b5646f8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:44.597748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-96caa9e5-19a9-45eb-be5e-b292b5646f8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:44.698290 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96caa9e5-19a9-45eb-be5e-b292b5646f8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:44.699927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96caa9e5-19a9-45eb-be5e-b292b5646f8f service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:44.720274 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 517/1037] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:47.117817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10425501-453c-43ab-a4b0-66182f9e16bc None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:47.118435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10425501-453c-43ab-a4b0-66182f9e16bc None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:47.118435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10425501-453c-43ab-a4b0-66182f9e16bc None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:47.125059 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 521/1038] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:47.126286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6b0f3-8ea3-43a5-bc59-cf1d4be5855f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:47.126784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6b0f3-8ea3-43a5-bc59-cf1d4be5855f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:47.126784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6b0f3-8ea3-43a5-bc59-cf1d4be5855f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:47.202999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e7c6b0f3-8ea3-43a5-bc59-cf1d4be5855f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:47.204213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e7c6b0f3-8ea3-43a5-bc59-cf1d4be5855f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:47.249168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:47.314219 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 518/1039] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:34:47.603064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10425501-453c-43ab-a4b0-66182f9e16bc None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:47.604593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-10425501-453c-43ab-a4b0-66182f9e16bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:47.791013 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10425501-453c-43ab-a4b0-66182f9e16bc None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:47.803100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:47.938528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454a4b4-a332-462f-873d-e8261f1d207b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:47.939136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a4b4-a332-462f-873d-e8261f1d207b service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:47.939417 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a4b4-a332-462f-873d-e8261f1d207b service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:47.939685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a4b4-a332-462f-873d-e8261f1d207b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:47.941276 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9454a4b4-a332-462f-873d-e8261f1d207b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:48.034586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454a4b4-a332-462f-873d-e8261f1d207b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:48.036090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454a4b4-a332-462f-873d-e8261f1d207b service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:48.054843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 522/1040] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:50.567939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c138b22-18e2-4558-ba7a-32f334e02d49 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:50.568477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c138b22-18e2-4558-ba7a-32f334e02d49 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:50.568477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c138b22-18e2-4558-ba7a-32f334e02d49 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:50.569968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 519/1041] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:50.575138 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340c6f-f3be-4d03-8ba3-ff01ac3fab2e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:50.575523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340c6f-f3be-4d03-8ba3-ff01ac3fab2e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:50.575722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340c6f-f3be-4d03-8ba3-ff01ac3fab2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:50.591979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:50.644892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f0340c6f-f3be-4d03-8ba3-ff01ac3fab2e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:50.646084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f0340c6f-f3be-4d03-8ba3-ff01ac3fab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:50.749871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 523/1042] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:51.045744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2973c-f10d-4c16-ae81-6d732e640ada None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:51.046222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2973c-f10d-4c16-ae81-6d732e640ada None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:51.046852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2973c-f10d-4c16-ae81-6d732e640ada None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:51.048496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 520/1043] 10.222.0.20 () {54 vars in 961 bytes} [Mon Oct 24 13:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:51.053929 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005bb6c-4995-476a-b729-b5b52aa544dd None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:51.054244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005bb6c-4995-476a-b729-b5b52aa544dd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:51.054517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005bb6c-4995-476a-b729-b5b52aa544dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:51.135552 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a005bb6c-4995-476a-b729-b5b52aa544dd None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:51.136885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a005bb6c-4995-476a-b729-b5b52aa544dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:51.175166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:51.272541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 524/1044] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Oct 24 13:34:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:51.280687 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2973c-f10d-4c16-ae81-6d732e640ada None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:51.282361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0bb2973c-f10d-4c16-ae81-6d732e640ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:51.437096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2973c-f10d-4c16-ae81-6d732e640ada None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:51.457051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:51.554379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a2851-d5f7-4a83-a03f-c1924fc59130 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:51.554986 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a2851-d5f7-4a83-a03f-c1924fc59130 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:51.555261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a2851-d5f7-4a83-a03f-c1924fc59130 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:51.555527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a2851-d5f7-4a83-a03f-c1924fc59130 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:51.557101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1f3a2851-d5f7-4a83-a03f-c1924fc59130 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:51.652258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a2851-d5f7-4a83-a03f-c1924fc59130 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:51.653858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a2851-d5f7-4a83-a03f-c1924fc59130 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:51.676398 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 521/1045] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:51.789980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c4c6-5e9f-4a43-be06-b37e4b0e63d5 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:51.790573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c4c6-5e9f-4a43-be06-b37e4b0e63d5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:51.790573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c4c6-5e9f-4a43-be06-b37e4b0e63d5 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:51.794564 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 525/1046] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:51.798483 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7238f4-b145-4c99-ae45-5a8d8aa8f45d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:51.798798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7238f4-b145-4c99-ae45-5a8d8aa8f45d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:51.799041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7238f4-b145-4c99-ae45-5a8d8aa8f45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:51.835598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:51.883787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2d7238f4-b145-4c99-ae45-5a8d8aa8f45d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:51.885036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2d7238f4-b145-4c99-ae45-5a8d8aa8f45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:52.006104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 522/1047] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:34:52.342356 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6338be1-a268-4c34-90d2-c4b55790f8fc None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:52.342744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6338be1-a268-4c34-90d2-c4b55790f8fc None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:52.343116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6338be1-a268-4c34-90d2-c4b55790f8fc None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:52.344699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 526/1048] 10.222.0.20 () {54 vars in 963 bytes} [Mon Oct 24 13:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:52.350159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7530c52-ce84-48f5-8b5c-0b218acc0997 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:52.350485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7530c52-ce84-48f5-8b5c-0b218acc0997 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:52.350756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7530c52-ce84-48f5-8b5c-0b218acc0997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:52.432929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c7530c52-ce84-48f5-8b5c-0b218acc0997 None None] MFA Rules not processed for user `3c8c594699c045a09c835910d894896b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:52.434114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c7530c52-ce84-48f5-8b5c-0b218acc0997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:52.490468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:52.524952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 523/1049] 10.222.0.20 () {58 vars in 1060 bytes} [Mon Oct 24 13:34:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:52.530634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6338be1-a268-4c34-90d2-c4b55790f8fc None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:52.532114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b6338be1-a268-4c34-90d2-c4b55790f8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:52.643134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6338be1-a268-4c34-90d2-c4b55790f8fc None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:52.696775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:52.736720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2e81e-b269-496b-b9a3-d415ce781d8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:52.737332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e81e-b269-496b-b9a3-d415ce781d8f service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:52.737613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e81e-b269-496b-b9a3-d415ce781d8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:52.737888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2e81e-b269-496b-b9a3-d415ce781d8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:52.739542 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5fb2e81e-b269-496b-b9a3-d415ce781d8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:52.869823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2e81e-b269-496b-b9a3-d415ce781d8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:52.871183 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2e81e-b269-496b-b9a3-d415ce781d8f service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:52.890531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 527/1050] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:34:53.930231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a089ba-d6aa-46ef-a520-42e7153cb618 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:53.930791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a089ba-d6aa-46ef-a520-42e7153cb618 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:53.930791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a089ba-d6aa-46ef-a520-42e7153cb618 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:53.932256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 524/1051] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:53.937631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ca643-8f93-41cb-a2aa-af6ab66980bb None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:53.938113 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ca643-8f93-41cb-a2aa-af6ab66980bb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:53.938248 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ca643-8f93-41cb-a2aa-af6ab66980bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.005095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-739ca643-8f93-41cb-a2aa-af6ab66980bb None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:54.006323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-739ca643-8f93-41cb-a2aa-af6ab66980bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:54.027751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:54.105928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 528/1052] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:54.408670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a089ba-d6aa-46ef-a520-42e7153cb618 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:54.410405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-42a089ba-d6aa-46ef-a520-42e7153cb618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:54.571849 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a089ba-d6aa-46ef-a520-42e7153cb618 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:54.625454 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:54.691532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe075b2b-0693-48bb-a9c9-35a988b4c6c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:54.692055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe075b2b-0693-48bb-a9c9-35a988b4c6c9 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.692333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe075b2b-0693-48bb-a9c9-35a988b4c6c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.692603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe075b2b-0693-48bb-a9c9-35a988b4c6c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.694218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fe075b2b-0693-48bb-a9c9-35a988b4c6c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:54.792232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe075b2b-0693-48bb-a9c9-35a988b4c6c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:54.793844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe075b2b-0693-48bb-a9c9-35a988b4c6c9 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:54.813107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 525/1053] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:57.054466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4974616-9901-4d84-918a-0cb303439759 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.055016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4974616-9901-4d84-918a-0cb303439759 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.055016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4974616-9901-4d84-918a-0cb303439759 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.056372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 529/1054] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:34:57.061832 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3013-25f2-4d48-8284-d613fbd430e1 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.062298 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3013-25f2-4d48-8284-d613fbd430e1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.062386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e3013-25f2-4d48-8284-d613fbd430e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.132336 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-980e3013-25f2-4d48-8284-d613fbd430e1 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:57.133514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-980e3013-25f2-4d48-8284-d613fbd430e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:57.141623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:57.250515 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 526/1055] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:57.543536 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4974616-9901-4d84-918a-0cb303439759 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:57.546780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c4974616-9901-4d84-918a-0cb303439759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:57.698068 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4974616-9901-4d84-918a-0cb303439759 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:57.703791 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:57.818973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c0571-0f32-46cb-9ac4-bff338b5be0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:57.819730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c0571-0f32-46cb-9ac4-bff338b5be0e service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.820102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c0571-0f32-46cb-9ac4-bff338b5be0e service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.820468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c0571-0f32-46cb-9ac4-bff338b5be0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.822125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b77c0571-0f32-46cb-9ac4-bff338b5be0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:57.929614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c0571-0f32-46cb-9ac4-bff338b5be0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:57.931354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c0571-0f32-46cb-9ac4-bff338b5be0e service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:57.950044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 530/1056] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:00.243146 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378e1ef-2b9b-49c7-a2ae-2d3986e0b4b8 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:00.243754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378e1ef-2b9b-49c7-a2ae-2d3986e0b4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:00.243754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378e1ef-2b9b-49c7-a2ae-2d3986e0b4b8 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:00.245165 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 527/1057] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:00.250938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179458c4-b88c-41eb-88fc-ea23a3094921 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:00.251405 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179458c4-b88c-41eb-88fc-ea23a3094921 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:00.251502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179458c4-b88c-41eb-88fc-ea23a3094921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:00.262808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:00.324678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-179458c4-b88c-41eb-88fc-ea23a3094921 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:00.325883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-179458c4-b88c-41eb-88fc-ea23a3094921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:00.438659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 531/1058] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:00.738419 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378e1ef-2b9b-49c7-a2ae-2d3986e0b4b8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:00.740385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1378e1ef-2b9b-49c7-a2ae-2d3986e0b4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:00.921995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:00.934786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378e1ef-2b9b-49c7-a2ae-2d3986e0b4b8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:01.077566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ce05e-949c-444e-b899-b8bf45b703ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:01.078193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ce05e-949c-444e-b899-b8bf45b703ff service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:01.078486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ce05e-949c-444e-b899-b8bf45b703ff service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:01.078757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ce05e-949c-444e-b899-b8bf45b703ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:01.080630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-301ce05e-949c-444e-b899-b8bf45b703ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:01.184862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ce05e-949c-444e-b899-b8bf45b703ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:01.186587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ce05e-949c-444e-b899-b8bf45b703ff service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:01.199825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:01.206194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 528/1059] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:03.469972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12d7c5-e1af-4d57-955b-6d3daec4dd71 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:03.470550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12d7c5-e1af-4d57-955b-6d3daec4dd71 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:03.470672 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12d7c5-e1af-4d57-955b-6d3daec4dd71 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:03.472720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 532/1060] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:35:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:03.479189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deecd7-7bcd-4269-8a8a-583167cf8985 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:03.479623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deecd7-7bcd-4269-8a8a-583167cf8985 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:03.479732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deecd7-7bcd-4269-8a8a-583167cf8985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:03.562776 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a4deecd7-7bcd-4269-8a8a-583167cf8985 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:03.564794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a4deecd7-7bcd-4269-8a8a-583167cf8985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:03.689071 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 529/1061] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:04.048464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12d7c5-e1af-4d57-955b-6d3daec4dd71 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:04.049966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0b12d7c5-e1af-4d57-955b-6d3daec4dd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:04.137308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:04.205742 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12d7c5-e1af-4d57-955b-6d3daec4dd71 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:04.340437 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4338e1-ebc0-4ce0-9b0d-3673bd7dbbe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:04.341207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4338e1-ebc0-4ce0-9b0d-3673bd7dbbe0 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:04.341916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4338e1-ebc0-4ce0-9b0d-3673bd7dbbe0 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:04.342624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4338e1-ebc0-4ce0-9b0d-3673bd7dbbe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:04.344775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8d4338e1-ebc0-4ce0-9b0d-3673bd7dbbe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:04.407961 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:04.456077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4338e1-ebc0-4ce0-9b0d-3673bd7dbbe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:04.457789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4338e1-ebc0-4ce0-9b0d-3673bd7dbbe0 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:04.476080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 533/1062] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:06.682822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74743e02-85d4-4860-8662-1df35f3a8701 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:06.683404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74743e02-85d4-4860-8662-1df35f3a8701 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:06.683404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74743e02-85d4-4860-8662-1df35f3a8701 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:06.684704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 530/1063] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:06.691050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c1cb2-f451-49fa-9db8-3ddbd9aa44b5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:06.691476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c1cb2-f451-49fa-9db8-3ddbd9aa44b5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:06.691596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c1cb2-f451-49fa-9db8-3ddbd9aa44b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:06.758602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ab6c1cb2-f451-49fa-9db8-3ddbd9aa44b5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:06.759772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ab6c1cb2-f451-49fa-9db8-3ddbd9aa44b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:06.857743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 534/1064] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:07.144278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74743e02-85d4-4860-8662-1df35f3a8701 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:07.145781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-74743e02-85d4-4860-8662-1df35f3a8701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:07.223615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:07.295222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74743e02-85d4-4860-8662-1df35f3a8701 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:07.409365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcc068-6e46-4f3d-8961-c585a278dcef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:07.409884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc068-6e46-4f3d-8961-c585a278dcef service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:07.410187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc068-6e46-4f3d-8961-c585a278dcef service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:07.410457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc068-6e46-4f3d-8961-c585a278dcef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:07.412068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3fdcc068-6e46-4f3d-8961-c585a278dcef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:07.452059 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:07.504156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcc068-6e46-4f3d-8961-c585a278dcef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:07.505738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcc068-6e46-4f3d-8961-c585a278dcef service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:07.524407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 531/1065] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:08.317925 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115ad4d-c505-42e8-9b04-e32f4abc5a01 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:08.318537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115ad4d-c505-42e8-9b04-e32f4abc5a01 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:08.318537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115ad4d-c505-42e8-9b04-e32f4abc5a01 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:08.319861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 535/1066] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:08.325342 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73975905-cc19-43aa-a176-00f92d7baa71 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:08.325901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73975905-cc19-43aa-a176-00f92d7baa71 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:08.326258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73975905-cc19-43aa-a176-00f92d7baa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:08.393925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-73975905-cc19-43aa-a176-00f92d7baa71 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:08.395157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-73975905-cc19-43aa-a176-00f92d7baa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:08.495573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 532/1067] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:08.798828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b5531-07a5-48e5-b4b2-f086e661ff5f None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:08.799110 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b5531-07a5-48e5-b4b2-f086e661ff5f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:08.799377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b5531-07a5-48e5-b4b2-f086e661ff5f None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:08.800700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 536/1068] 10.222.0.20 () {54 vars in 962 bytes} [Mon Oct 24 13:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:08.806238 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5e98c-3981-4069-9173-5b1924f78949 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:08.806527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5e98c-3981-4069-9173-5b1924f78949 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:08.806803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5e98c-3981-4069-9173-5b1924f78949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:08.822645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:08.885020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a9a5e98c-3981-4069-9173-5b1924f78949 None None] MFA Rules not processed for user `d07f8443d5594f47bc81e9f9dc9ce43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:08.886273 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a9a5e98c-3981-4069-9173-5b1924f78949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:08.962479 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 533/1069] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Oct 24 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:35:08.971000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b5531-07a5-48e5-b4b2-f086e661ff5f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:08.972541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-904b5531-07a5-48e5-b4b2-f086e661ff5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:09.003442 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:09.080756 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b5531-07a5-48e5-b4b2-f086e661ff5f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:09.155778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63edef-6604-425a-ac5f-f084a6a674b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07f8443d5594f47bc81e9f9dc9ce43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:09.156382 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63edef-6604-425a-ac5f-f084a6a674b3 service glance] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:09.156658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63edef-6604-425a-ac5f-f084a6a674b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:09.156923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63edef-6604-425a-ac5f-f084a6a674b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:09.158533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1d63edef-6604-425a-ac5f-f084a6a674b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:09.240931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:09.273591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63edef-6604-425a-ac5f-f084a6a674b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:09.275058 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63edef-6604-425a-ac5f-f084a6a674b3 service glance] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:09.293643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 537/1070] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:09.724235 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750db31-1d2c-4178-bfe7-fb5439f19df3 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:09.724778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750db31-1d2c-4178-bfe7-fb5439f19df3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:09.724778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750db31-1d2c-4178-bfe7-fb5439f19df3 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:09.726033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 534/1071] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:09.731563 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07517be3-f7d2-4623-94ec-f8dc0f1eb7b4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:09.733356 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07517be3-f7d2-4623-94ec-f8dc0f1eb7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:09.733643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07517be3-f7d2-4623-94ec-f8dc0f1eb7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:09.802097 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-07517be3-f7d2-4623-94ec-f8dc0f1eb7b4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:09.803295 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-07517be3-f7d2-4623-94ec-f8dc0f1eb7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:09.912935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 538/1072] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:10.055943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a1b37-f0f6-4ed3-8b6f-55fadeaaa7cb None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:10.056230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a1b37-f0f6-4ed3-8b6f-55fadeaaa7cb None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:10.056650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a1b37-f0f6-4ed3-8b6f-55fadeaaa7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:10.133582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-128a1b37-f0f6-4ed3-8b6f-55fadeaaa7cb None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:10.134919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-128a1b37-f0f6-4ed3-8b6f-55fadeaaa7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:10.211305 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07517be3-f7d2-4623-94ec-f8dc0f1eb7b4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:10.211997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-07517be3-f7d2-4623-94ec-f8dc0f1eb7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:10.217577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:10.241055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 535/1073] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:35:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:10.259890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:10.287617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d2c8e-9b3d-419a-a35f-f857a2d8a740 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:10.287990 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d2c8e-9b3d-419a-a35f-f857a2d8a740 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:10.288322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d2c8e-9b3d-419a-a35f-f857a2d8a740 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:10.290033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 536/1074] 10.222.0.20 () {54 vars in 943 bytes} [Mon Oct 24 13:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:35:10.296353 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bb16d-63aa-4f8e-ad0e-953df598f722 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:10.296705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bb16d-63aa-4f8e-ad0e-953df598f722 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:10.297110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bb16d-63aa-4f8e-ad0e-953df598f722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:10.339495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07517be3-f7d2-4623-94ec-f8dc0f1eb7b4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:10.376155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-eb8bb16d-63aa-4f8e-ad0e-953df598f722 None None] MFA Rules not processed for user `0ea0d300a51c41fba4d4193dd2251a50`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:10.377393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-eb8bb16d-63aa-4f8e-ad0e-953df598f722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:10.457410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2c654-efbf-4ee8-bdd5-7a2cf343be81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:10.458016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2c654-efbf-4ee8-bdd5-7a2cf343be81 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:10.458399 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2c654-efbf-4ee8-bdd5-7a2cf343be81 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:10.458665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2c654-efbf-4ee8-bdd5-7a2cf343be81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:10.460239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a1d2c654-efbf-4ee8-bdd5-7a2cf343be81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:10.490682 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 537/1075] 10.222.0.20 () {58 vars in 1040 bytes} [Mon Oct 24 13:35:10 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:35:10.494225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:10.501244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bb16d-63aa-4f8e-ad0e-953df598f722 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:10.511690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bb16d-63aa-4f8e-ad0e-953df598f722 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:10.537694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:10.551934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2c654-efbf-4ee8-bdd5-7a2cf343be81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:10.553504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2c654-efbf-4ee8-bdd5-7a2cf343be81 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:10.572332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 539/1076] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:10.623454 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b2317-7b33-417d-953c-007381b7aa39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:10.624065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b2317-7b33-417d-953c-007381b7aa39 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:10.624337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b2317-7b33-417d-953c-007381b7aa39 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:10.624604 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b2317-7b33-417d-953c-007381b7aa39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:10.626432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-be8b2317-7b33-417d-953c-007381b7aa39 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:10.720014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b2317-7b33-417d-953c-007381b7aa39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:10.721657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b2317-7b33-417d-953c-007381b7aa39 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:10.731096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 538/1077] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:35:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:35:12.814722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878770fd-e3b1-4f45-993e-38cc88980a19 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:12.815707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878770fd-e3b1-4f45-993e-38cc88980a19 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:12.816507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878770fd-e3b1-4f45-993e-38cc88980a19 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:12.818189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 540/1078] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:35:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:12.823986 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a7656-116d-4df6-8aef-0eca394475a8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:12.824458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a7656-116d-4df6-8aef-0eca394475a8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:12.824550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a7656-116d-4df6-8aef-0eca394475a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:12.908871 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1a9a7656-116d-4df6-8aef-0eca394475a8 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:12.910416 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1a9a7656-116d-4df6-8aef-0eca394475a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:12.917590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:13.026214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 539/1079] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:13.336594 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878770fd-e3b1-4f45-993e-38cc88980a19 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:13.338105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-878770fd-e3b1-4f45-993e-38cc88980a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:13.508294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878770fd-e3b1-4f45-993e-38cc88980a19 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:13.521307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:13.645884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c0af2-7642-4d16-bf9c-1784227d3c63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:13.646539 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c0af2-7642-4d16-bf9c-1784227d3c63 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:13.646817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c0af2-7642-4d16-bf9c-1784227d3c63 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:13.647087 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c0af2-7642-4d16-bf9c-1784227d3c63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:13.648692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-503c0af2-7642-4d16-bf9c-1784227d3c63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:13.748420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c0af2-7642-4d16-bf9c-1784227d3c63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:13.750063 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c0af2-7642-4d16-bf9c-1784227d3c63 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:13.769735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 541/1080] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:16.013959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef71e35-adef-4d6b-ba58-1286b055e0b9 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:16.015097 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef71e35-adef-4d6b-ba58-1286b055e0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:16.015437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef71e35-adef-4d6b-ba58-1286b055e0b9 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:16.016962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 540/1081] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:35:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:16.022205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50708d44-b68c-4d9d-8d39-a529a2ed0c3b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:16.022638 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50708d44-b68c-4d9d-8d39-a529a2ed0c3b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:16.022772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50708d44-b68c-4d9d-8d39-a529a2ed0c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:16.045492 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:16.094680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-50708d44-b68c-4d9d-8d39-a529a2ed0c3b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:16.095928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-50708d44-b68c-4d9d-8d39-a529a2ed0c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:16.216912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 542/1082] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:16.510601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef71e35-adef-4d6b-ba58-1286b055e0b9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:16.512103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1ef71e35-adef-4d6b-ba58-1286b055e0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:16.662089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:16.675182 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef71e35-adef-4d6b-ba58-1286b055e0b9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:16.798563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339f88a-4c3a-4f5d-80d3-92071385c15a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:16.799057 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339f88a-4c3a-4f5d-80d3-92071385c15a service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:16.799355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339f88a-4c3a-4f5d-80d3-92071385c15a service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:16.799627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339f88a-4c3a-4f5d-80d3-92071385c15a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:16.801215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7339f88a-4c3a-4f5d-80d3-92071385c15a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:16.897524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339f88a-4c3a-4f5d-80d3-92071385c15a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:16.899135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339f88a-4c3a-4f5d-80d3-92071385c15a service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:16.910920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:16.923176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 541/1083] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:19.233161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e597d73-2f46-4812-bce6-17fa2cece37d None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:19.233161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e597d73-2f46-4812-bce6-17fa2cece37d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:19.233161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e597d73-2f46-4812-bce6-17fa2cece37d None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:19.233161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 543/1084] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:35:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:19.237670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3615-5ecc-4a39-a696-d286bd6c13e6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:19.238064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3615-5ecc-4a39-a696-d286bd6c13e6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:19.238247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3615-5ecc-4a39-a696-d286bd6c13e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:19.310178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-049e3615-5ecc-4a39-a696-d286bd6c13e6 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:19.311399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-049e3615-5ecc-4a39-a696-d286bd6c13e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:19.420442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 542/1085] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:19.713466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e597d73-2f46-4812-bce6-17fa2cece37d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:19.715032 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9e597d73-2f46-4812-bce6-17fa2cece37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:19.818992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:19.877618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e597d73-2f46-4812-bce6-17fa2cece37d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:20.000035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f8db4-8e5f-4866-bf68-acd53367f02b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:20.000567 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f8db4-8e5f-4866-bf68-acd53367f02b service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:20.000901 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f8db4-8e5f-4866-bf68-acd53367f02b service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:20.001112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f8db4-8e5f-4866-bf68-acd53367f02b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:20.002766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-557f8db4-8e5f-4866-bf68-acd53367f02b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:20.067913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:20.102999 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f8db4-8e5f-4866-bf68-acd53367f02b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:20.104570 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f8db4-8e5f-4866-bf68-acd53367f02b service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:20.126296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 544/1086] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:29.418552 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=67 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:30.124555 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=20 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:34.133014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf487000-987f-42af-8f5b-627b5ba898e2 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:34.133564 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf487000-987f-42af-8f5b-627b5ba898e2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:34.133564 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf487000-987f-42af-8f5b-627b5ba898e2 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:34.134899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 543/1087] 10.222.0.20 () {54 vars in 918 bytes} [Mon Oct 24 13:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:34.140312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e3d36-3a23-433c-b3e0-bf817541dd1f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:34.140758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e3d36-3a23-433c-b3e0-bf817541dd1f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:34.140866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e3d36-3a23-433c-b3e0-bf817541dd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:34.216213 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-2a3e3d36-3a23-433c-b3e0-bf817541dd1f None None] MFA Rules not processed for user `e0ce1753e22d4952ab5d4348b0cffc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:34.218075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2a3e3d36-3a23-433c-b3e0-bf817541dd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:34.323197 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 545/1088] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Oct 24 13:35:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:34.337615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46c0c-390b-4460-b7b7-a83f9294a693 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:34.337897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46c0c-390b-4460-b7b7-a83f9294a693 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:34.338190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46c0c-390b-4460-b7b7-a83f9294a693 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:34.339520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 544/1089] 10.222.0.20 () {54 vars in 918 bytes} [Mon Oct 24 13:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:34.344850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b5f7b-f2b0-4f80-a70c-fcfde5f22eb1 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:34.345132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b5f7b-f2b0-4f80-a70c-fcfde5f22eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:34.345407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b5f7b-f2b0-4f80-a70c-fcfde5f22eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:34.418490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:34.421537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-182b5f7b-f2b0-4f80-a70c-fcfde5f22eb1 None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:34.423342 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-182b5f7b-f2b0-4f80-a70c-fcfde5f22eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:34.537385 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 546/1090] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Oct 24 13:35:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:37.140669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18295e96-6bd7-4f81-b167-f749c5b1cea5 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:37.141239 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18295e96-6bd7-4f81-b167-f749c5b1cea5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:37.141239 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18295e96-6bd7-4f81-b167-f749c5b1cea5 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:37.142562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 545/1091] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:37.148077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c9fa-e00b-4cdc-9331-9343350b3779 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:37.148507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c9fa-e00b-4cdc-9331-9343350b3779 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:37.148678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c9fa-e00b-4cdc-9331-9343350b3779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:37.215806 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-25c9c9fa-e00b-4cdc-9331-9343350b3779 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:37.216997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-25c9c9fa-e00b-4cdc-9331-9343350b3779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:37.272043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:37.314781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 547/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:37.605678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18295e96-6bd7-4f81-b167-f749c5b1cea5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:37.607281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-18295e96-6bd7-4f81-b167-f749c5b1cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:37.750292 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18295e96-6bd7-4f81-b167-f749c5b1cea5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:37.821387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:37.862390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6f39f-0721-4c9e-8d06-9b0d35b76cea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:37.862922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6f39f-0721-4c9e-8d06-9b0d35b76cea service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:37.863229 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6f39f-0721-4c9e-8d06-9b0d35b76cea service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:37.863500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6f39f-0721-4c9e-8d06-9b0d35b76cea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:37.865084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-50c6f39f-0721-4c9e-8d06-9b0d35b76cea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:37.954968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6f39f-0721-4c9e-8d06-9b0d35b76cea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:37.956548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6f39f-0721-4c9e-8d06-9b0d35b76cea service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:37.974245 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 546/1093] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:39.116348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27060c5b-ba3a-4f60-a4f2-829790cfd320 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:39.116916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27060c5b-ba3a-4f60-a4f2-829790cfd320 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:39.116916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27060c5b-ba3a-4f60-a4f2-829790cfd320 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:39.118231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 548/1094] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:39.123197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621997b5-fa8e-4bce-8c89-40c8a9c7c6ff None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:39.123663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621997b5-fa8e-4bce-8c89-40c8a9c7c6ff None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:39.123839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621997b5-fa8e-4bce-8c89-40c8a9c7c6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:39.202420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-621997b5-fa8e-4bce-8c89-40c8a9c7c6ff None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:39.203618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-621997b5-fa8e-4bce-8c89-40c8a9c7c6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:39.230861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:39.301852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 547/1095] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:39.455849 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27060c5b-ba3a-4f60-a4f2-829790cfd320 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:39.457269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-27060c5b-ba3a-4f60-a4f2-829790cfd320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:39.601220 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27060c5b-ba3a-4f60-a4f2-829790cfd320 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:39.644909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:39.717896 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68958bfe-2711-454f-80a8-8300cd3c82ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:39.718440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68958bfe-2711-454f-80a8-8300cd3c82ba service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:39.718713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68958bfe-2711-454f-80a8-8300cd3c82ba service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:39.718981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68958bfe-2711-454f-80a8-8300cd3c82ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:39.720546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-68958bfe-2711-454f-80a8-8300cd3c82ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:39.808258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68958bfe-2711-454f-80a8-8300cd3c82ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:39.809844 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68958bfe-2711-454f-80a8-8300cd3c82ba service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:39.828401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 549/1096] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:40.951119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0707b-c40d-427b-ac2d-6bb02cae9799 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:40.952362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0707b-c40d-427b-ac2d-6bb02cae9799 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:40.952362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0707b-c40d-427b-ac2d-6bb02cae9799 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:40.953669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 548/1097] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:35:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:40.958505 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acd927-684a-4a1b-82f4-e18881bced6c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:40.958918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acd927-684a-4a1b-82f4-e18881bced6c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:40.959060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acd927-684a-4a1b-82f4-e18881bced6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:41.012712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:41.035299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-15acd927-684a-4a1b-82f4-e18881bced6c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:41.036473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-15acd927-684a-4a1b-82f4-e18881bced6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:41.131348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 550/1098] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:41.282766 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0707b-c40d-427b-ac2d-6bb02cae9799 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:41.284196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-17b0707b-c40d-427b-ac2d-6bb02cae9799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:41.427798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0707b-c40d-427b-ac2d-6bb02cae9799 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:41.440341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:41.542098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f44af2-4fd4-43a2-a846-7df40ef7e867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:41.542645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f44af2-4fd4-43a2-a846-7df40ef7e867 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:41.542925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f44af2-4fd4-43a2-a846-7df40ef7e867 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:41.543196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f44af2-4fd4-43a2-a846-7df40ef7e867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:41.544783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f6f44af2-4fd4-43a2-a846-7df40ef7e867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:41.632701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f44af2-4fd4-43a2-a846-7df40ef7e867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:41.634288 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f44af2-4fd4-43a2-a846-7df40ef7e867 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:41.651525 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 549/1099] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:42.744092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab1637-d60e-4b8d-b2f4-687c0562dcd8 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:42.744698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab1637-d60e-4b8d-b2f4-687c0562dcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:42.744698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab1637-d60e-4b8d-b2f4-687c0562dcd8 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:42.745943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 551/1100] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:42.751437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870acaa5-7855-45d8-962e-728757a325d9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:42.751867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870acaa5-7855-45d8-962e-728757a325d9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:42.751989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870acaa5-7855-45d8-962e-728757a325d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:42.771661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:42.828503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-870acaa5-7855-45d8-962e-728757a325d9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:42.829701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-870acaa5-7855-45d8-962e-728757a325d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:42.925033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 550/1101] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:43.074959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab1637-d60e-4b8d-b2f4-687c0562dcd8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:43.076517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-66ab1637-d60e-4b8d-b2f4-687c0562dcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:43.187321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:43.224933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab1637-d60e-4b8d-b2f4-687c0562dcd8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:43.334996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae7698-1cf3-4977-a650-f18c16e96fc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:43.335596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae7698-1cf3-4977-a650-f18c16e96fc4 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:43.335875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae7698-1cf3-4977-a650-f18c16e96fc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:43.336142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae7698-1cf3-4977-a650-f18c16e96fc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:43.337712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-11ae7698-1cf3-4977-a650-f18c16e96fc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:43.413590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:43.427399 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae7698-1cf3-4977-a650-f18c16e96fc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:43.428962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae7698-1cf3-4977-a650-f18c16e96fc4 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:43.446519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 552/1102] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:44.549922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2d814-05b5-4723-9963-7dc1aec4910b None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:44.550516 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2d814-05b5-4723-9963-7dc1aec4910b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:44.550516 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2d814-05b5-4723-9963-7dc1aec4910b None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:44.551871 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 551/1103] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:44.556828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473bfce-009a-4d9d-8ccb-ee912e6ecff7 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:44.557242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473bfce-009a-4d9d-8ccb-ee912e6ecff7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:44.557354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473bfce-009a-4d9d-8ccb-ee912e6ecff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:44.631157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7473bfce-009a-4d9d-8ccb-ee912e6ecff7 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:44.632332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7473bfce-009a-4d9d-8ccb-ee912e6ecff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:44.726451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 553/1104] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:44.880490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2d814-05b5-4723-9963-7dc1aec4910b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:44.881931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-20c2d814-05b5-4723-9963-7dc1aec4910b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:44.966033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:45.028319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2d814-05b5-4723-9963-7dc1aec4910b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:45.140942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c35c0-efd5-4225-824f-6cddfa5983b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:45.141457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c35c0-efd5-4225-824f-6cddfa5983b3 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:45.141735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c35c0-efd5-4225-824f-6cddfa5983b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:45.142001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c35c0-efd5-4225-824f-6cddfa5983b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:45.143622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-534c35c0-efd5-4225-824f-6cddfa5983b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:45.191341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:45.238046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c35c0-efd5-4225-824f-6cddfa5983b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:45.239661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c35c0-efd5-4225-824f-6cddfa5983b3 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:45.257117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 552/1105] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:46.333275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a19251-8197-4258-81cf-356c381301b6 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:46.333914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a19251-8197-4258-81cf-356c381301b6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:46.333914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a19251-8197-4258-81cf-356c381301b6 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:46.335295 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 554/1106] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:46.340430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e871c0b-4e2e-4bc5-b411-505d57659d07 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:46.340871 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e871c0b-4e2e-4bc5-b411-505d57659d07 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:46.340980 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e871c0b-4e2e-4bc5-b411-505d57659d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:46.417157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0e871c0b-4e2e-4bc5-b411-505d57659d07 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:46.418387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0e871c0b-4e2e-4bc5-b411-505d57659d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:46.512793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 553/1107] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:46.662664 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a19251-8197-4258-81cf-356c381301b6 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:46.664489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d4a19251-8197-4258-81cf-356c381301b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:46.713236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:46.812042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a19251-8197-4258-81cf-356c381301b6 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:46.924848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24a05e-a432-4df0-b595-33e8ebdc12d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:46.925446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24a05e-a432-4df0-b595-33e8ebdc12d3 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:46.925719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24a05e-a432-4df0-b595-33e8ebdc12d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:46.925985 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24a05e-a432-4df0-b595-33e8ebdc12d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:46.927593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-df24a05e-a432-4df0-b595-33e8ebdc12d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:46.934284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:47.017706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24a05e-a432-4df0-b595-33e8ebdc12d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:47.019299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24a05e-a432-4df0-b595-33e8ebdc12d3 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:47.036808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 555/1108] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:48.081068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d667956-f824-4150-9dd6-9b860ce4251b None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:48.081632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d667956-f824-4150-9dd6-9b860ce4251b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:48.081632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d667956-f824-4150-9dd6-9b860ce4251b None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:48.083002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 554/1109] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:48.087938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8b33-c43c-4a7e-a820-a63e40bc01ef None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:48.088347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8b33-c43c-4a7e-a820-a63e40bc01ef None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:48.088513 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8b33-c43c-4a7e-a820-a63e40bc01ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:48.164096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-befb8b33-c43c-4a7e-a820-a63e40bc01ef None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:48.165192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-befb8b33-c43c-4a7e-a820-a63e40bc01ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:48.233589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:48.262039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 556/1110] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:48.413646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d667956-f824-4150-9dd6-9b860ce4251b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:48.415275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3d667956-f824-4150-9dd6-9b860ce4251b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:48.432381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:48.559592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d667956-f824-4150-9dd6-9b860ce4251b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:48.647895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:48.673857 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4349ad-3c47-4d44-bf7e-fcea25cae2ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:48.674418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4349ad-3c47-4d44-bf7e-fcea25cae2ac service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:48.674693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4349ad-3c47-4d44-bf7e-fcea25cae2ac service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:48.674957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4349ad-3c47-4d44-bf7e-fcea25cae2ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:48.676541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8e4349ad-3c47-4d44-bf7e-fcea25cae2ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:48.765169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4349ad-3c47-4d44-bf7e-fcea25cae2ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:48.766770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4349ad-3c47-4d44-bf7e-fcea25cae2ac service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:48.784308 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 555/1111] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:49.832481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe580c-e83f-4990-a413-9809be6e161b None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:49.833049 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe580c-e83f-4990-a413-9809be6e161b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:49.833049 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe580c-e83f-4990-a413-9809be6e161b None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:49.834428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 557/1112] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:49.839389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e459562-e02e-4df5-b4d1-75f0a3ce4a5b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:49.839820 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e459562-e02e-4df5-b4d1-75f0a3ce4a5b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:49.839937 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e459562-e02e-4df5-b4d1-75f0a3ce4a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:49.915275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4e459562-e02e-4df5-b4d1-75f0a3ce4a5b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:49.916466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4e459562-e02e-4df5-b4d1-75f0a3ce4a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:49.958244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:50.013248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 556/1113] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:50.174981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe580c-e83f-4990-a413-9809be6e161b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:50.176414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1dfe580c-e83f-4990-a413-9809be6e161b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:50.323892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe580c-e83f-4990-a413-9809be6e161b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:50.377754 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:50.437089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06bb6c-84b3-4f91-8b47-874c0f916838 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:50.437627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06bb6c-84b3-4f91-8b47-874c0f916838 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:50.437906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06bb6c-84b3-4f91-8b47-874c0f916838 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:50.438206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06bb6c-84b3-4f91-8b47-874c0f916838 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:50.439793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4f06bb6c-84b3-4f91-8b47-874c0f916838 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:50.527652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06bb6c-84b3-4f91-8b47-874c0f916838 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:50.530283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06bb6c-84b3-4f91-8b47-874c0f916838 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:50.549098 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 558/1114] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:51.592639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bd5fe-6ef1-450f-a2bc-91cf305ebfb6 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:51.592639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bd5fe-6ef1-450f-a2bc-91cf305ebfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:51.593329 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bd5fe-6ef1-450f-a2bc-91cf305ebfb6 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:51.594316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 557/1115] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:51.599277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a0541-eb4c-4d34-9282-3d05b914ceba None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:51.599716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a0541-eb4c-4d34-9282-3d05b914ceba None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:51.599827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a0541-eb4c-4d34-9282-3d05b914ceba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:51.675142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4c7a0541-eb4c-4d34-9282-3d05b914ceba None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:51.676325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4c7a0541-eb4c-4d34-9282-3d05b914ceba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:51.678375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:51.773461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 559/1116] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:51.924308 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bd5fe-6ef1-450f-a2bc-91cf305ebfb6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:51.925849 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6e9bd5fe-6ef1-450f-a2bc-91cf305ebfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:52.070189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bd5fe-6ef1-450f-a2bc-91cf305ebfb6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:52.097209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:52.181781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549cbe7-81c8-43ff-9c97-891937b1cafa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:52.182326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549cbe7-81c8-43ff-9c97-891937b1cafa service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:52.182604 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549cbe7-81c8-43ff-9c97-891937b1cafa service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:52.182877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549cbe7-81c8-43ff-9c97-891937b1cafa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:52.184450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c549cbe7-81c8-43ff-9c97-891937b1cafa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:52.274695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549cbe7-81c8-43ff-9c97-891937b1cafa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:52.276288 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549cbe7-81c8-43ff-9c97-891937b1cafa service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:52.293876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 558/1117] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:54.386915 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3826d-522b-4af0-9ad3-2b52ed8b17cb None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:54.387580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3826d-522b-4af0-9ad3-2b52ed8b17cb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:54.387580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3826d-522b-4af0-9ad3-2b52ed8b17cb None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:54.388991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 560/1118] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:54.394726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3acc4-e6a4-4ffe-9b23-71a489944ad7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:54.395193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3acc4-e6a4-4ffe-9b23-71a489944ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:54.395302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3acc4-e6a4-4ffe-9b23-71a489944ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:54.437016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:54.462643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-aca3acc4-e6a4-4ffe-9b23-71a489944ad7 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:54.463770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-aca3acc4-e6a4-4ffe-9b23-71a489944ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:54.560693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 559/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:35:54.852666 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3826d-522b-4af0-9ad3-2b52ed8b17cb None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:54.854225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-31f3826d-522b-4af0-9ad3-2b52ed8b17cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:54.956052 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3826d-522b-4af0-9ad3-2b52ed8b17cb None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:54.988469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:55.029701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4661c-3079-4ad5-b75a-6a362c3215e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:55.030337 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4661c-3079-4ad5-b75a-6a362c3215e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:55.030612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4661c-3079-4ad5-b75a-6a362c3215e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:55.030876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4661c-3079-4ad5-b75a-6a362c3215e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:55.032450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-89a4661c-3079-4ad5-b75a-6a362c3215e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:55.140876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4661c-3079-4ad5-b75a-6a362c3215e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:55.142268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4661c-3079-4ad5-b75a-6a362c3215e3 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:55.160374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 561/1120] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:57.450390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3c8a5-0e65-4f0b-a867-655e9d14fff9 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:57.451564 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3c8a5-0e65-4f0b-a867-655e9d14fff9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:57.451564 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3c8a5-0e65-4f0b-a867-655e9d14fff9 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:57.452808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 560/1121] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:35:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:35:57.457983 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13187f3-cfd2-4b1a-8f45-4b1c3107e44f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:57.458447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13187f3-cfd2-4b1a-8f45-4b1c3107e44f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:57.458579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13187f3-cfd2-4b1a-8f45-4b1c3107e44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:57.524256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a13187f3-cfd2-4b1a-8f45-4b1c3107e44f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:57.525367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a13187f3-cfd2-4b1a-8f45-4b1c3107e44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:57.538209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:57.621994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 562/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:57.914565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3c8a5-0e65-4f0b-a867-655e9d14fff9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:57.916096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c9c3c8a5-0e65-4f0b-a867-655e9d14fff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:58.018423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3c8a5-0e65-4f0b-a867-655e9d14fff9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:58.031518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:58.092705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0593d-d763-4add-9e8f-d0e43b6e33d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:58.093302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0593d-d763-4add-9e8f-d0e43b6e33d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:58.093580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0593d-d763-4add-9e8f-d0e43b6e33d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:58.093848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0593d-d763-4add-9e8f-d0e43b6e33d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:58.095454 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b5d0593d-d763-4add-9e8f-d0e43b6e33d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:58.203095 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0593d-d763-4add-9e8f-d0e43b6e33d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:58.204445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0593d-d763-4add-9e8f-d0e43b6e33d4 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:58.223220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 561/1123] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:00.694592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d2899-b976-4434-8f09-56191779056b None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:00.695149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d2899-b976-4434-8f09-56191779056b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:00.695149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d2899-b976-4434-8f09-56191779056b None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:00.696462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 563/1124] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:00.701893 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb01787-08c3-4627-8051-5f574d9bfc06 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:00.702304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb01787-08c3-4627-8051-5f574d9bfc06 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:00.702599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb01787-08c3-4627-8051-5f574d9bfc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:00.750988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:00.769124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bdb01787-08c3-4627-8051-5f574d9bfc06 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:00.770349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bdb01787-08c3-4627-8051-5f574d9bfc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:00.867430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 562/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:36:01.159156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d2899-b976-4434-8f09-56191779056b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:01.160704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-da5d2899-b976-4434-8f09-56191779056b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:01.274767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d2899-b976-4434-8f09-56191779056b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:01.317329 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:01.353918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff1854-e695-4f65-8506-32bb789ca67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:01.354614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff1854-e695-4f65-8506-32bb789ca67b service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:01.354889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff1854-e695-4f65-8506-32bb789ca67b service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:01.355154 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff1854-e695-4f65-8506-32bb789ca67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:01.356792 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cbff1854-e695-4f65-8506-32bb789ca67b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:01.476617 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff1854-e695-4f65-8506-32bb789ca67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:01.477953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff1854-e695-4f65-8506-32bb789ca67b service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:01.497291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 564/1126] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:03.802721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c1f47-3a2f-4a1f-84f0-58d8ae534f66 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:03.803291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c1f47-3a2f-4a1f-84f0-58d8ae534f66 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:03.803291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c1f47-3a2f-4a1f-84f0-58d8ae534f66 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:03.804634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 563/1127] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:03.810728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4598af9-8f2c-4efb-9cea-42ebd830451d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:03.811370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4598af9-8f2c-4efb-9cea-42ebd830451d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:03.811490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4598af9-8f2c-4efb-9cea-42ebd830451d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:03.876954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b4598af9-8f2c-4efb-9cea-42ebd830451d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:03.878140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b4598af9-8f2c-4efb-9cea-42ebd830451d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:03.898597 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:03.977069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 565/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:04.263509 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c1f47-3a2f-4a1f-84f0-58d8ae534f66 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:04.265122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-070c1f47-3a2f-4a1f-84f0-58d8ae534f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:04.368526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c1f47-3a2f-4a1f-84f0-58d8ae534f66 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:04.389362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:04.440990 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0936a-7391-4700-99fe-4c2c2c1485cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:04.441587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0936a-7391-4700-99fe-4c2c2c1485cb service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:04.441863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0936a-7391-4700-99fe-4c2c2c1485cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:04.442136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0936a-7391-4700-99fe-4c2c2c1485cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:04.443734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-33c0936a-7391-4700-99fe-4c2c2c1485cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:04.551695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0936a-7391-4700-99fe-4c2c2c1485cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:04.553106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0936a-7391-4700-99fe-4c2c2c1485cb service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:04.571244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 564/1129] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:06.784873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b95f5c-4bf1-4817-9db3-889f7c5601e7 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:06.785447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b95f5c-4bf1-4817-9db3-889f7c5601e7 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:06.785447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b95f5c-4bf1-4817-9db3-889f7c5601e7 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:06.786766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 566/1130] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:06.792239 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19822f-cc2f-43a3-a6f6-82d06c08128d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:06.792676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19822f-cc2f-43a3-a6f6-82d06c08128d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:06.792792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19822f-cc2f-43a3-a6f6-82d06c08128d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:06.855263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:06.858699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ab19822f-cc2f-43a3-a6f6-82d06c08128d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:06.860538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ab19822f-cc2f-43a3-a6f6-82d06c08128d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:06.956741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 565/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:07.243983 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b95f5c-4bf1-4817-9db3-889f7c5601e7 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:07.245591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-61b95f5c-4bf1-4817-9db3-889f7c5601e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:07.348379 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b95f5c-4bf1-4817-9db3-889f7c5601e7 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:07.398553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:07.423738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c8aea-eaf1-4c32-9fcd-82ff57686048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:07.424337 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c8aea-eaf1-4c32-9fcd-82ff57686048 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:07.424610 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c8aea-eaf1-4c32-9fcd-82ff57686048 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:07.424873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c8aea-eaf1-4c32-9fcd-82ff57686048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:07.426466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7d5c8aea-eaf1-4c32-9fcd-82ff57686048 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:07.533699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c8aea-eaf1-4c32-9fcd-82ff57686048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:07.535091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c8aea-eaf1-4c32-9fcd-82ff57686048 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:07.552778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 567/1132] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:09.754303 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d09926-60c2-4295-9fef-dc315346d7e7 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:09.754913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d09926-60c2-4295-9fef-dc315346d7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:09.754913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d09926-60c2-4295-9fef-dc315346d7e7 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:09.756173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 566/1133] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:09.761377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e58265-a22d-4c55-92da-987c99f442fe None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:09.761807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e58265-a22d-4c55-92da-987c99f442fe None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:09.761988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e58265-a22d-4c55-92da-987c99f442fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:09.827467 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-72e58265-a22d-4c55-92da-987c99f442fe None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:09.828645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-72e58265-a22d-4c55-92da-987c99f442fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:09.856183 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:09.924906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 568/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:10.208216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d09926-60c2-4295-9fef-dc315346d7e7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:10.209783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-31d09926-60c2-4295-9fef-dc315346d7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:10.311909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d09926-60c2-4295-9fef-dc315346d7e7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:10.339901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:10.385177 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7c486-bcee-4056-aa3c-5082c5d47cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:10.385776 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7c486-bcee-4056-aa3c-5082c5d47cae service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:10.386051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7c486-bcee-4056-aa3c-5082c5d47cae service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:10.386349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7c486-bcee-4056-aa3c-5082c5d47cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:10.388422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1aa7c486-bcee-4056-aa3c-5082c5d47cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:10.494647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7c486-bcee-4056-aa3c-5082c5d47cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:10.495987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7c486-bcee-4056-aa3c-5082c5d47cae service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:10.514563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 567/1135] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:12.733249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b4eb8-2c8a-4d5c-b61c-4a70dfaf407a None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:12.733819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b4eb8-2c8a-4d5c-b61c-4a70dfaf407a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:12.733819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b4eb8-2c8a-4d5c-b61c-4a70dfaf407a None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:12.735159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 569/1136] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:12.740603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9f08-0667-47d9-81be-0fa177f90d51 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:12.741029 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9f08-0667-47d9-81be-0fa177f90d51 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:12.741147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9f08-0667-47d9-81be-0fa177f90d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:12.806568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-781e9f08-0667-47d9-81be-0fa177f90d51 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:12.807760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-781e9f08-0667-47d9-81be-0fa177f90d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:12.815942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:12.905651 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 568/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:13.195884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b4eb8-2c8a-4d5c-b61c-4a70dfaf407a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:13.197516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-dd7b4eb8-2c8a-4d5c-b61c-4a70dfaf407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:13.302084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b4eb8-2c8a-4d5c-b61c-4a70dfaf407a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:13.359426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:13.377876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d7049-3500-45d9-8347-e43dfa800d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:13.378506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d7049-3500-45d9-8347-e43dfa800d6e service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:13.378783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d7049-3500-45d9-8347-e43dfa800d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:13.379050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d7049-3500-45d9-8347-e43dfa800d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:13.380625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6a9d7049-3500-45d9-8347-e43dfa800d6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:13.488633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d7049-3500-45d9-8347-e43dfa800d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:13.490047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d7049-3500-45d9-8347-e43dfa800d6e service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:13.508672 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 570/1138] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:15.716118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1798d-4064-43b8-b718-dfc8bffd2d37 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:15.716686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1798d-4064-43b8-b718-dfc8bffd2d37 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:15.716686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1798d-4064-43b8-b718-dfc8bffd2d37 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:15.718187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 569/1139] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:15.723613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93136dc0-3548-49a2-afdf-7a2f4b1f9624 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:15.724043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93136dc0-3548-49a2-afdf-7a2f4b1f9624 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:15.724187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93136dc0-3548-49a2-afdf-7a2f4b1f9624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:15.789831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-93136dc0-3548-49a2-afdf-7a2f4b1f9624 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:15.790968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-93136dc0-3548-49a2-afdf-7a2f4b1f9624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:15.829356 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:15.887637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 571/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:16.173736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1798d-4064-43b8-b718-dfc8bffd2d37 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:16.175374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5ce1798d-4064-43b8-b718-dfc8bffd2d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:16.281215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1798d-4064-43b8-b718-dfc8bffd2d37 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:16.319188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:16.355344 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963560c9-6b10-4a88-8bb7-79e7a270a103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:16.356031 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963560c9-6b10-4a88-8bb7-79e7a270a103 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:16.356308 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963560c9-6b10-4a88-8bb7-79e7a270a103 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:16.357518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963560c9-6b10-4a88-8bb7-79e7a270a103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:16.359153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-963560c9-6b10-4a88-8bb7-79e7a270a103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:16.466425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963560c9-6b10-4a88-8bb7-79e7a270a103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:16.467744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963560c9-6b10-4a88-8bb7-79e7a270a103 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:16.485594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 570/1141] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:18.656507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c31744-c891-4354-9774-925a2be43057 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:18.657048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c31744-c891-4354-9774-925a2be43057 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:18.657048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c31744-c891-4354-9774-925a2be43057 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:18.658412 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 572/1142] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:18.664043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84616d-8048-47fc-9601-de25e7c22083 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:18.664507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84616d-8048-47fc-9601-de25e7c22083 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:18.664606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84616d-8048-47fc-9601-de25e7c22083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:18.729221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1d84616d-8048-47fc-9601-de25e7c22083 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:18.730429 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1d84616d-8048-47fc-9601-de25e7c22083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:18.748337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:18.828960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 571/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:36:19.116439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c31744-c891-4354-9774-925a2be43057 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:19.117998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-02c31744-c891-4354-9774-925a2be43057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:19.223691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c31744-c891-4354-9774-925a2be43057 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:19.291916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:19.299015 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ee676-367c-40eb-bea4-c7e3ef79533b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:19.299641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ee676-367c-40eb-bea4-c7e3ef79533b service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:19.299897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ee676-367c-40eb-bea4-c7e3ef79533b service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:19.300163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ee676-367c-40eb-bea4-c7e3ef79533b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:19.301742 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d68ee676-367c-40eb-bea4-c7e3ef79533b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:19.411517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ee676-367c-40eb-bea4-c7e3ef79533b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:19.413102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ee676-367c-40eb-bea4-c7e3ef79533b service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:19.431340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 573/1144] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:21.620230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f2cdf-e3d5-47cc-b430-76d387f661af None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:21.620833 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f2cdf-e3d5-47cc-b430-76d387f661af None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:21.620833 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f2cdf-e3d5-47cc-b430-76d387f661af None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:21.622330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 572/1145] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:21.627603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d38fa-4704-4890-be63-5c03cb20f747 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:21.627998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d38fa-4704-4890-be63-5c03cb20f747 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:21.628178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d38fa-4704-4890-be63-5c03cb20f747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:21.695223 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0f3d38fa-4704-4890-be63-5c03cb20f747 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:21.696423 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0f3d38fa-4704-4890-be63-5c03cb20f747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:21.743932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:21.792720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 574/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:22.072881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f2cdf-e3d5-47cc-b430-76d387f661af None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:22.074508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fd0f2cdf-e3d5-47cc-b430-76d387f661af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:22.176830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f2cdf-e3d5-47cc-b430-76d387f661af None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:22.225789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:22.253418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedc591-0230-4b34-830f-c16dc5bb056b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:22.254016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc591-0230-4b34-830f-c16dc5bb056b service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:22.254324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc591-0230-4b34-830f-c16dc5bb056b service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:22.254591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc591-0230-4b34-830f-c16dc5bb056b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:22.256226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8eedc591-0230-4b34-830f-c16dc5bb056b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:22.364970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedc591-0230-4b34-830f-c16dc5bb056b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:22.366406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedc591-0230-4b34-830f-c16dc5bb056b service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:22.384840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 573/1147] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:31.791176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=22 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:32.382628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=66 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:43.139297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459f4f-b835-4359-a3a0-c2918d64095b None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:43.139883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459f4f-b835-4359-a3a0-c2918d64095b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:43.139883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08459f4f-b835-4359-a3a0-c2918d64095b None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:43.141171 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 575/1148] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:36:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:43.146667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c40a-4c4e-4cc4-801c-865429e63c55 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:43.147090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c40a-4c4e-4cc4-801c-865429e63c55 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:43.147202 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62c40a-4c4e-4cc4-801c-865429e63c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:43.213940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4e62c40a-4c4e-4cc4-801c-865429e63c55 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:43.215173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4e62c40a-4c4e-4cc4-801c-865429e63c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:43.312481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 574/1149] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:43.620854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca20cd-795d-47e3-97a4-9b8506307258 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:43.621013 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca20cd-795d-47e3-97a4-9b8506307258 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:43.621230 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca20cd-795d-47e3-97a4-9b8506307258 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:43.623852 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 576/1150] 10.222.0.20 () {54 vars in 962 bytes} [Mon Oct 24 13:36:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:43.629862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252c576-a8cd-4225-b4b5-8d35bf8dc724 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:43.630205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252c576-a8cd-4225-b4b5-8d35bf8dc724 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:43.630468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252c576-a8cd-4225-b4b5-8d35bf8dc724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:43.705278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a252c576-a8cd-4225-b4b5-8d35bf8dc724 None None] MFA Rules not processed for user `d07f8443d5594f47bc81e9f9dc9ce43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:43.706554 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a252c576-a8cd-4225-b4b5-8d35bf8dc724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:43.727026 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:43.783039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 575/1151] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Oct 24 13:36:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:43.791224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca20cd-795d-47e3-97a4-9b8506307258 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:43.792759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-eaca20cd-795d-47e3-97a4-9b8506307258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:43.897062 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca20cd-795d-47e3-97a4-9b8506307258 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:43.973544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa341d7-3a55-4b59-81d0-a22960374ec8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07f8443d5594f47bc81e9f9dc9ce43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:43.974147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa341d7-3a55-4b59-81d0-a22960374ec8 service glance] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:43.974449 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa341d7-3a55-4b59-81d0-a22960374ec8 service glance] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:43.974715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa341d7-3a55-4b59-81d0-a22960374ec8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:43.976291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7fa341d7-3a55-4b59-81d0-a22960374ec8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:44.031528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:44.087207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa341d7-3a55-4b59-81d0-a22960374ec8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:44.088616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa341d7-3a55-4b59-81d0-a22960374ec8 service glance] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:44.106812 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 577/1152] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:36:46.352749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31583300-1427-48b7-978a-283009e5d4ea None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:46.353309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31583300-1427-48b7-978a-283009e5d4ea None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:46.353309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31583300-1427-48b7-978a-283009e5d4ea None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:46.354852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 576/1153] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:46.360624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932c04-1e23-4ca8-96c8-aad61d7fbbab None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:46.361051 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932c04-1e23-4ca8-96c8-aad61d7fbbab None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:46.361186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd932c04-1e23-4ca8-96c8-aad61d7fbbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:46.430139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-dd932c04-1e23-4ca8-96c8-aad61d7fbbab None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:46.431365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-dd932c04-1e23-4ca8-96c8-aad61d7fbbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:46.532311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 578/1154] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:46.807598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31583300-1427-48b7-978a-283009e5d4ea None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:46.809135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-31583300-1427-48b7-978a-283009e5d4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:46.934673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31583300-1427-48b7-978a-283009e5d4ea None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:46.983009 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:47.027692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3defc5-f7be-4625-b23e-f1679ee78f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:47.028294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3defc5-f7be-4625-b23e-f1679ee78f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:47.028569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3defc5-f7be-4625-b23e-f1679ee78f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:47.028841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3defc5-f7be-4625-b23e-f1679ee78f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:47.029466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3defc5-f7be-4625-b23e-f1679ee78f98 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:47.030875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-2b3defc5-f7be-4625-b23e-f1679ee78f98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:36:47.030875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Oct 24 13:36:47.030875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: Oct 24 13:36:47.032046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3defc5-f7be-4625-b23e-f1679ee78f98 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:47.051444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 577/1155] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Oct 24 13:36:46 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:36:49.273797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9c3c6-17db-4eee-905b-05ddda1c4936 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:49.274331 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9c3c6-17db-4eee-905b-05ddda1c4936 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:49.274453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9c3c6-17db-4eee-905b-05ddda1c4936 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:49.275696 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 579/1156] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:49.281909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8d5af-e303-4e9a-b5c5-4edabe36c6b9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:49.282401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8d5af-e303-4e9a-b5c5-4edabe36c6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:49.282550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8d5af-e303-4e9a-b5c5-4edabe36c6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:49.349313 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9de8d5af-e303-4e9a-b5c5-4edabe36c6b9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:49.350532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9de8d5af-e303-4e9a-b5c5-4edabe36c6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:49.447286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 578/1157] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:49.751189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9c3c6-17db-4eee-905b-05ddda1c4936 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:49.752721 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9ba9c3c6-17db-4eee-905b-05ddda1c4936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:49.762570 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:49.898958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9c3c6-17db-4eee-905b-05ddda1c4936 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:49.977853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:50.011362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d8b97-f2d3-46b7-bebf-7732ede9ba3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:50.011892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d8b97-f2d3-46b7-bebf-7732ede9ba3f service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:50.012165 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d8b97-f2d3-46b7-bebf-7732ede9ba3f service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:50.012428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d8b97-f2d3-46b7-bebf-7732ede9ba3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:50.014008 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7c3d8b97-f2d3-46b7-bebf-7732ede9ba3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:50.104195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d8b97-f2d3-46b7-bebf-7732ede9ba3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:50.105772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d8b97-f2d3-46b7-bebf-7732ede9ba3f service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:50.124325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 580/1158] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:52.496160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5027d5f-4d83-4362-8000-520bf4669f03 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:52.496740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5027d5f-4d83-4362-8000-520bf4669f03 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:52.496740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5027d5f-4d83-4362-8000-520bf4669f03 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:52.498007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 579/1159] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:52.504318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b773-1ea1-42c9-a137-7b976b385177 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:52.504739 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b773-1ea1-42c9-a137-7b976b385177 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:52.504845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b773-1ea1-42c9-a137-7b976b385177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:52.571102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5511b773-1ea1-42c9-a137-7b976b385177 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:52.572295 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5511b773-1ea1-42c9-a137-7b976b385177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:52.617777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:52.668590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 581/1160] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:52.962985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5027d5f-4d83-4362-8000-520bf4669f03 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:52.964511 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d5027d5f-4d83-4362-8000-520bf4669f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:52.989230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:53.068179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5027d5f-4d83-4362-8000-520bf4669f03 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:53.140364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420347c9-490d-4ab1-a8cf-7d356c5bef47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:53.140956 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420347c9-490d-4ab1-a8cf-7d356c5bef47 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:53.141230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420347c9-490d-4ab1-a8cf-7d356c5bef47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:53.141495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420347c9-490d-4ab1-a8cf-7d356c5bef47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:53.143113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-420347c9-490d-4ab1-a8cf-7d356c5bef47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:53.219339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:53.254920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420347c9-490d-4ab1-a8cf-7d356c5bef47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:53.256348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420347c9-490d-4ab1-a8cf-7d356c5bef47 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:53.274735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 580/1161] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:55.476801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f5657-acdd-45d4-a23b-35dc3dba3682 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:55.477336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f5657-acdd-45d4-a23b-35dc3dba3682 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:55.477336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f5657-acdd-45d4-a23b-35dc3dba3682 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:55.478653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 582/1162] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:36:55.484917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15650280-a908-4da2-b0c8-c4759340b08d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:55.485309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15650280-a908-4da2-b0c8-c4759340b08d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:55.485455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15650280-a908-4da2-b0c8-c4759340b08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:55.553909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-15650280-a908-4da2-b0c8-c4759340b08d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:55.555143 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-15650280-a908-4da2-b0c8-c4759340b08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:55.650607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 581/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:55.942237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f5657-acdd-45d4-a23b-35dc3dba3682 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:55.943809 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9d9f5657-acdd-45d4-a23b-35dc3dba3682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:56.044573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f5657-acdd-45d4-a23b-35dc3dba3682 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:56.115463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:56.117024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c1b28-4908-4ad9-8668-ce4696535a73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:56.117686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c1b28-4908-4ad9-8668-ce4696535a73 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:56.117973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c1b28-4908-4ad9-8668-ce4696535a73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:56.118512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c1b28-4908-4ad9-8668-ce4696535a73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:56.120461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-346c1b28-4908-4ad9-8668-ce4696535a73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:56.230082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c1b28-4908-4ad9-8668-ce4696535a73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:56.231538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c1b28-4908-4ad9-8668-ce4696535a73 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:56.249690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 583/1164] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:58.563227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5988ff5-f3a0-4747-8ce9-4fd81cc58fd1 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:58.563789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5988ff5-f3a0-4747-8ce9-4fd81cc58fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:58.563789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5988ff5-f3a0-4747-8ce9-4fd81cc58fd1 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:58.565129 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 582/1165] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:36:58.570478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a1888-a10e-40b5-8862-da0edc724fed None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:58.570874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a1888-a10e-40b5-8862-da0edc724fed None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:58.571023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a1888-a10e-40b5-8862-da0edc724fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:58.636934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b06a1888-a10e-40b5-8862-da0edc724fed None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:58.638127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b06a1888-a10e-40b5-8862-da0edc724fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:58.692039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:58.736306 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 584/1166] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:59.029521 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5988ff5-f3a0-4747-8ce9-4fd81cc58fd1 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:59.031541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e5988ff5-f3a0-4747-8ce9-4fd81cc58fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:59.068020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:59.135253 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5988ff5-f3a0-4747-8ce9-4fd81cc58fd1 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:59.207616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcffd4-9587-4d16-85a0-27d4a691b190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:59.208216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcffd4-9587-4d16-85a0-27d4a691b190 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:59.208491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcffd4-9587-4d16-85a0-27d4a691b190 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:59.208758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcffd4-9587-4d16-85a0-27d4a691b190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:59.210361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a2fcffd4-9587-4d16-85a0-27d4a691b190 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:59.291166 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:59.321468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcffd4-9587-4d16-85a0-27d4a691b190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:59.322924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcffd4-9587-4d16-85a0-27d4a691b190 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:59.340520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 583/1167] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:01.706400 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ecfe8-4b0b-417f-954e-da038a2fee32 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:01.706945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ecfe8-4b0b-417f-954e-da038a2fee32 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:01.706945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ecfe8-4b0b-417f-954e-da038a2fee32 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:01.708295 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 585/1168] 10.222.0.20 () {54 vars in 927 bytes} [Mon Oct 24 13:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:37:01.713739 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0ab92-b44b-4eb1-b740-49c9a5de44ef None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:01.714194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0ab92-b44b-4eb1-b740-49c9a5de44ef None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:01.714307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0ab92-b44b-4eb1-b740-49c9a5de44ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:01.779426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-40e0ab92-b44b-4eb1-b740-49c9a5de44ef None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:01.780972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-40e0ab92-b44b-4eb1-b740-49c9a5de44ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:01.882198 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 584/1169] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Oct 24 13:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:02.176245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ecfe8-4b0b-417f-954e-da038a2fee32 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:02.177773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-260ecfe8-4b0b-417f-954e-da038a2fee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:02.283319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ecfe8-4b0b-417f-954e-da038a2fee32 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:02.360260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4cbde-29aa-42d5-b7de-28b0b7647a4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:02.360863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4cbde-29aa-42d5-b7de-28b0b7647a4a service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:02.361141 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4cbde-29aa-42d5-b7de-28b0b7647a4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:02.361398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4cbde-29aa-42d5-b7de-28b0b7647a4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:02.363111 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e0f4cbde-29aa-42d5-b7de-28b0b7647a4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:02.372473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:02.478002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4cbde-29aa-42d5-b7de-28b0b7647a4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:02.479361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4cbde-29aa-42d5-b7de-28b0b7647a4a service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:02.498056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 586/1170] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:11.876904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=82 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:12.495793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=50 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:15.969786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15660a-0329-4da0-affd-db070b0b329d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:15.970265 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15660a-0329-4da0-affd-db070b0b329d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:15.970602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15660a-0329-4da0-affd-db070b0b329d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:16.048753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-df15660a-0329-4da0-affd-db070b0b329d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:38:16.049943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-df15660a-0329-4da0-affd-db070b0b329d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:16.146662 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 585/1171] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:38:16.154673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bec75f-8b74-461b-b91b-dc2a8f0c71b4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:16.155177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bec75f-8b74-461b-b91b-dc2a8f0c71b4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:16.155284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bec75f-8b74-461b-b91b-dc2a8f0c71b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:16.234763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-70bec75f-8b74-461b-b91b-dc2a8f0c71b4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:38:16.236076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-70bec75f-8b74-461b-b91b-dc2a8f0c71b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:16.333649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 587/1172] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:38:16.338350 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15660a-0329-4da0-affd-db070b0b329d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:16.339543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-df15660a-0329-4da0-affd-db070b0b329d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:16.350938 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15660a-0329-4da0-affd-db070b0b329d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:16.412459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:16.447964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3159b-9e2c-4ca8-b62b-711bf865a75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:16.448565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3159b-9e2c-4ca8-b62b-711bf865a75e admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:16.448840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3159b-9e2c-4ca8-b62b-711bf865a75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:16.449107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3159b-9e2c-4ca8-b62b-711bf865a75e admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:16.449762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3159b-9e2c-4ca8-b62b-711bf865a75e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:38:16.450060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3159b-9e2c-4ca8-b62b-711bf865a75e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:16.451099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-a8e3159b-9e2c-4ca8-b62b-711bf865a75e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:38:16.451099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Oct 24 13:38:16.451099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: Oct 24 13:38:16.451977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3159b-9e2c-4ca8-b62b-711bf865a75e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:16.462888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 586/1173] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:38:16.472766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bec75f-8b74-461b-b91b-dc2a8f0c71b4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:16.482559 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bec75f-8b74-461b-b91b-dc2a8f0c71b4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:16.542945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:16.577021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210cc68-3854-42b1-be68-c69dff7d0a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:16.577620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210cc68-3854-42b1-be68-c69dff7d0a79 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:16.577898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210cc68-3854-42b1-be68-c69dff7d0a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:16.578197 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210cc68-3854-42b1-be68-c69dff7d0a79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:16.578813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210cc68-3854-42b1-be68-c69dff7d0a79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:16.579844 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-7210cc68-3854-42b1-be68-c69dff7d0a79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:38:16.579844 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Oct 24 13:38:16.579844 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: Oct 24 13:38:16.581025 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210cc68-3854-42b1-be68-c69dff7d0a79 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:16.636081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 588/1174] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:38:16 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:38:16.640578 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3159b-9e2c-4ca8-b62b-711bf865a75e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:16.642091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a8e3159b-9e2c-4ca8-b62b-711bf865a75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:16.676654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3159b-9e2c-4ca8-b62b-711bf865a75e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:16.769248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacc06d-11e0-4b1b-9b30-5f71abc4a59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:16.769844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacc06d-11e0-4b1b-9b30-5f71abc4a59c admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:16.770123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacc06d-11e0-4b1b-9b30-5f71abc4a59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:16.770426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacc06d-11e0-4b1b-9b30-5f71abc4a59c admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:16.771155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cacc06d-11e0-4b1b-9b30-5f71abc4a59c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:16.772159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-7cacc06d-11e0-4b1b-9b30-5f71abc4a59c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:38:16.772159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Oct 24 13:38:16.772159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: Oct 24 13:38:16.773362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cacc06d-11e0-4b1b-9b30-5f71abc4a59c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:16.814930 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 587/1175] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:38:16 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:38:16.819055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210cc68-3854-42b1-be68-c69dff7d0a79 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:16.820537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7210cc68-3854-42b1-be68-c69dff7d0a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:16.882791 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210cc68-3854-42b1-be68-c69dff7d0a79 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:16.966272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:16.976951 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28c7e7-7d37-4270-9eaa-55f2f094d116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:16.977550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28c7e7-7d37-4270-9eaa-55f2f094d116 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:16.977825 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28c7e7-7d37-4270-9eaa-55f2f094d116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:16.978088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28c7e7-7d37-4270-9eaa-55f2f094d116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:16.978747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28c7e7-7d37-4270-9eaa-55f2f094d116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:16.980343 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28c7e7-7d37-4270-9eaa-55f2f094d116 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:16.993076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 589/1176] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:38:16.997360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacc06d-11e0-4b1b-9b30-5f71abc4a59c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:16.998893 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7cacc06d-11e0-4b1b-9b30-5f71abc4a59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:17.009858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacc06d-11e0-4b1b-9b30-5f71abc4a59c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:17.034782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:17.105264 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c165d-90a1-4a3e-a4d4-a0a456afc87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:17.105829 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c165d-90a1-4a3e-a4d4-a0a456afc87f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:17.106100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c165d-90a1-4a3e-a4d4-a0a456afc87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:17.106416 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c165d-90a1-4a3e-a4d4-a0a456afc87f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:17.107037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c165d-90a1-4a3e-a4d4-a0a456afc87f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:17.108597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c165d-90a1-4a3e-a4d4-a0a456afc87f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:17.120554 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 588/1177] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:38:17.124866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28c7e7-7d37-4270-9eaa-55f2f094d116 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:17.126424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ba28c7e7-7d37-4270-9eaa-55f2f094d116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:17.159943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28c7e7-7d37-4270-9eaa-55f2f094d116 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:17.253511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bebb48-9ca0-4e59-b53b-56015fecff40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:17.254158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bebb48-9ca0-4e59-b53b-56015fecff40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:17.254473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bebb48-9ca0-4e59-b53b-56015fecff40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:17.254719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bebb48-9ca0-4e59-b53b-56015fecff40 admin admin] PATH_INFO: `/v3/projects/755471c23dd44427b39aeed30dc11cb2/users/d5cbc505d8324ab68e0a7597ff3bd10e/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:17.277286 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bebb48-9ca0-4e59-b53b-56015fecff40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755471c23dd44427b39aeed30dc11cb2, user_id=d5cbc505d8324ab68e0a7597ff3bd10e, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:17.280552 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bebb48-9ca0-4e59-b53b-56015fecff40 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:17.305700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 590/1178] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:38:17 2022] PUT /identity/v3/projects/755471c23dd44427b39aeed30dc11cb2/users/d5cbc505d8324ab68e0a7597ff3bd10e/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:38:17.316898 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c4ef5-4d54-4065-9a9c-238d7e78c49f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:17.317176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c4ef5-4d54-4065-9a9c-238d7e78c49f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:17.317447 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c4ef5-4d54-4065-9a9c-238d7e78c49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:17.374720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bb5c4ef5-4d54-4065-9a9c-238d7e78c49f None None] MFA Rules not processed for user `d5cbc505d8324ab68e0a7597ff3bd10e`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:38:17.375847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bb5c4ef5-4d54-4065-9a9c-238d7e78c49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:17.453280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 589/1179] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:38:17.466588 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ae48f-c9d5-4313-a28b-2a8e8b68b0fd None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:17.466879 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ae48f-c9d5-4313-a28b-2a8e8b68b0fd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:17.467158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ae48f-c9d5-4313-a28b-2a8e8b68b0fd None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:17.468487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 591/1180] 10.222.0.20 () {54 vars in 963 bytes} [Mon Oct 24 13:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:38:17.473244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9706d-3750-4934-b17d-78499d217844 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:17.473529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9706d-3750-4934-b17d-78499d217844 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:17.473803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9706d-3750-4934-b17d-78499d217844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:17.505888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:17.551840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-91c9706d-3750-4934-b17d-78499d217844 None None] MFA Rules not processed for user `3c8c594699c045a09c835910d894896b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:38:17.553020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-91c9706d-3750-4934-b17d-78499d217844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:17.628625 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 590/1181] 10.222.0.20 () {58 vars in 1060 bytes} [Mon Oct 24 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:38:17.636287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ae48f-c9d5-4313-a28b-2a8e8b68b0fd None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:17.637816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cf4ae48f-c9d5-4313-a28b-2a8e8b68b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:17.688023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:17.741873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ae48f-c9d5-4313-a28b-2a8e8b68b0fd None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:17.816196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec8205-9194-4bd9-bf16-7ea7a2a67d2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:17.816793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec8205-9194-4bd9-bf16-7ea7a2a67d2b service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:17.817072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec8205-9194-4bd9-bf16-7ea7a2a67d2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:17.817342 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec8205-9194-4bd9-bf16-7ea7a2a67d2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:17.818962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-18ec8205-9194-4bd9-bf16-7ea7a2a67d2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:17.923373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec8205-9194-4bd9-bf16-7ea7a2a67d2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:17.924745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec8205-9194-4bd9-bf16-7ea7a2a67d2b service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:17.928182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:17.942906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 592/1182] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:38:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:38:17.989313 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e903d0-5a3b-4d40-8fd5-f85f70ac8102 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:17.989597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e903d0-5a3b-4d40-8fd5-f85f70ac8102 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:17.989868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e903d0-5a3b-4d40-8fd5-f85f70ac8102 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:17.991161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 591/1183] 10.222.0.20 () {52 vars in 830 bytes} [Mon Oct 24 13:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:38:17.994712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec8205-9194-4bd9-bf16-7ea7a2a67d2b service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:17.995915 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-18ec8205-9194-4bd9-bf16-7ea7a2a67d2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:18.096699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec8205-9194-4bd9-bf16-7ea7a2a67d2b service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:18.191678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:18.192313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:18.192490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:18.192755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] PATH_INFO: `/v3/users/d5cbc505d8324ab68e0a7597ff3bd10e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:18.206133 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5cbc505d8324ab68e0a7597ff3bd10e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:18.207244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:38:18.207244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Oct 24 13:38:18.207244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: Oct 24 13:38:18.208513 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:18.219361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:18.274385 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5cbc505d8324ab68e0a7597ff3bd10e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:38:18.274918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5cbc505d8324ab68e0a7597ff3bd10e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:38:18.298517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5cbc505d8324ab68e0a7597ff3bd10e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:38:18.298968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5cbc505d8324ab68e0a7597ff3bd10e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:38:18.313238 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5cbc505d8324ab68e0a7597ff3bd10e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:38:18.327906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 593/1184] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:38:17 2022] DELETE /identity/v3/users/d5cbc505d8324ab68e0a7597ff3bd10e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:38:18.332568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85f659-a6fe-4fb2-87ba-a48ed6139575 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:18.333050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85f659-a6fe-4fb2-87ba-a48ed6139575 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:18.333176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85f659-a6fe-4fb2-87ba-a48ed6139575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:18.390300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2b85f659-a6fe-4fb2-87ba-a48ed6139575 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:38:18.391524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2b85f659-a6fe-4fb2-87ba-a48ed6139575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:18.467172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:18.479254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 592/1185] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:38:18.490480 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:18.491698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:18.590832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2008607-1f06-41f3-b3d6-279cc7942f10 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:18.647551 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:18.668002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd03dbd-7f53-402d-96cb-10b3a49162ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:18.668602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd03dbd-7f53-402d-96cb-10b3a49162ad service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:18.668876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd03dbd-7f53-402d-96cb-10b3a49162ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:18.669142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd03dbd-7f53-402d-96cb-10b3a49162ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:18.671123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ddd03dbd-7f53-402d-96cb-10b3a49162ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:18.778691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd03dbd-7f53-402d-96cb-10b3a49162ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:18.780939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd03dbd-7f53-402d-96cb-10b3a49162ad service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:18.798877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 594/1186] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:38:19.117209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85f659-a6fe-4fb2-87ba-a48ed6139575 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:19.118468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2b85f659-a6fe-4fb2-87ba-a48ed6139575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:19.130103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85f659-a6fe-4fb2-87ba-a48ed6139575 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:19.229911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:19.230565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:19.230841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:19.231104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] PATH_INFO: `/v3/projects/755471c23dd44427b39aeed30dc11cb2` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:19.241047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=755471c23dd44427b39aeed30dc11cb2)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:19.242052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:38:19.242052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Oct 24 13:38:19.242052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: Oct 24 13:38:19.243266 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:19.303460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755471c23dd44427b39aeed30dc11cb2', 'request_id': 'req-ba57df0e-ae69-4106-bc1e-2f62034a6342'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:38:19.314176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755471c23dd44427b39aeed30dc11cb2', 'request_id': 'req-ba57df0e-ae69-4106-bc1e-2f62034a6342'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:38:19.328633 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755471c23dd44427b39aeed30dc11cb2', 'request_id': 'req-ba57df0e-ae69-4106-bc1e-2f62034a6342'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:38:19.350734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] The token cache is being invalidate because project 755471c23dd44427b39aeed30dc11cb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:38:19.351388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:38:19.351499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ba57df0e-ae69-4106-bc1e-2f62034a6342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:38:19.352790 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 593/1187] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:38:19 2022] DELETE /identity/v3/projects/755471c23dd44427b39aeed30dc11cb2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:38:28.796695 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=62 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:29.348639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:39:46.502510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4181e-e51d-47e2-93e4-c46c850418a8 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:46.503118 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4181e-e51d-47e2-93e4-c46c850418a8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:46.503118 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4181e-e51d-47e2-93e4-c46c850418a8 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:46.504402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 595/1188] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:39:46.509398 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e754b-e456-44eb-828d-5b87abe165fb None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:46.509803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e754b-e456-44eb-828d-5b87abe165fb None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:46.509936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e754b-e456-44eb-828d-5b87abe165fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:46.590450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c49e754b-e456-44eb-828d-5b87abe165fb None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:39:46.591639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c49e754b-e456-44eb-828d-5b87abe165fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:39:46.696769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 594/1189] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:39:46.855814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4181e-e51d-47e2-93e4-c46c850418a8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:39:46.857939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-30f4181e-e51d-47e2-93e4-c46c850418a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:39:46.979823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4181e-e51d-47e2-93e4-c46c850418a8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:39:46.986089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb0ab6-338b-4fc7-9f2e-71376b861a87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:39:46.986733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb0ab6-338b-4fc7-9f2e-71376b861a87 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:46.987007 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb0ab6-338b-4fc7-9f2e-71376b861a87 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:46.987275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb0ab6-338b-4fc7-9f2e-71376b861a87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:46.995298 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb0ab6-338b-4fc7-9f2e-71376b861a87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:39:46.996863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb0ab6-338b-4fc7-9f2e-71376b861a87 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:39:47.006724 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 596/1190] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:39:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:39:48.100068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fa1ea-678f-4436-87c8-302322b8714e None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:48.100591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fa1ea-678f-4436-87c8-302322b8714e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:48.100591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fa1ea-678f-4436-87c8-302322b8714e None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:48.101955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 595/1191] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:39:48.106637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca808c28-2183-4c88-9350-d986f1695b0e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:48.107090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca808c28-2183-4c88-9350-d986f1695b0e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:48.107201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca808c28-2183-4c88-9350-d986f1695b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:48.126941 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ca808c28-2183-4c88-9350-d986f1695b0e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:39:48.128106 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ca808c28-2183-4c88-9350-d986f1695b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:39:48.135828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 597/1192] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:39:48.287721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fa1ea-678f-4436-87c8-302322b8714e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:39:48.302697 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fa1ea-678f-4436-87c8-302322b8714e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:39:48.308145 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2a1d5-1488-4073-9c08-afd182e82e85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:39:48.308743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2a1d5-1488-4073-9c08-afd182e82e85 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:48.309025 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2a1d5-1488-4073-9c08-afd182e82e85 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:48.309299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2a1d5-1488-4073-9c08-afd182e82e85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:48.317686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2a1d5-1488-4073-9c08-afd182e82e85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:39:48.319298 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2a1d5-1488-4073-9c08-afd182e82e85 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:39:48.329159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 596/1193] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:39:48.374338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a584313-6176-40f4-88ad-325d38ce1d87 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:48.374988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a584313-6176-40f4-88ad-325d38ce1d87 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:48.375270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a584313-6176-40f4-88ad-325d38ce1d87 None None] PATH_INFO: `/v3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:48.376805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 598/1194] 10.222.0.45 () {54 vars in 925 bytes} [Mon Oct 24 13:39:48 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:39:48.383201 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb25177-8e43-4a50-b9f9-518a5ee65dbd None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:48.383477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb25177-8e43-4a50-b9f9-518a5ee65dbd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:48.383741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb25177-8e43-4a50-b9f9-518a5ee65dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:48.449062 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8fb25177-8e43-4a50-b9f9-518a5ee65dbd None None] MFA Rules not processed for user `e0ce1753e22d4952ab5d4348b0cffc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:39:48.450279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8fb25177-8e43-4a50-b9f9-518a5ee65dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:39:48.468904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:39:48.523691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 597/1195] 10.222.0.45 () {58 vars in 1015 bytes} [Mon Oct 24 13:39:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:39:52.065072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512a52f-0d87-4d89-a4ba-ceebc2443c54 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:52.065530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512a52f-0d87-4d89-a4ba-ceebc2443c54 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:52.065659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512a52f-0d87-4d89-a4ba-ceebc2443c54 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:52.067033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 599/1196] 10.222.0.20 () {54 vars in 961 bytes} [Mon Oct 24 13:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:39:52.072306 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f75661-83e6-4090-ac3d-d84268c5b6e6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:52.072737 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f75661-83e6-4090-ac3d-d84268c5b6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:52.072865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f75661-83e6-4090-ac3d-d84268c5b6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:52.092966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b3f75661-83e6-4090-ac3d-d84268c5b6e6 None None] MFA Rules not processed for user `e0ce1753e22d4952ab5d4348b0cffc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:39:52.094200 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b3f75661-83e6-4090-ac3d-d84268c5b6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:39:52.102732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 598/1197] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Oct 24 13:39:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:39:52.110641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512a52f-0d87-4d89-a4ba-ceebc2443c54 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:39:52.123819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512a52f-0d87-4d89-a4ba-ceebc2443c54 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:39:52.129345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737c6d6-aa0c-4f8d-a08a-d1f6af8fadfe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:39:52.129949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737c6d6-aa0c-4f8d-a08a-d1f6af8fadfe service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:52.130261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737c6d6-aa0c-4f8d-a08a-d1f6af8fadfe service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:52.130529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737c6d6-aa0c-4f8d-a08a-d1f6af8fadfe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:52.138418 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737c6d6-aa0c-4f8d-a08a-d1f6af8fadfe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:39:52.140732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737c6d6-aa0c-4f8d-a08a-d1f6af8fadfe service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:39:52.150471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 600/1198] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:39:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:39:59.782495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff276a7-b1d0-4d67-a1bf-06c46c57c4ca None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:59.783047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff276a7-b1d0-4d67-a1bf-06c46c57c4ca None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:59.783047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff276a7-b1d0-4d67-a1bf-06c46c57c4ca None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:59.784373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 599/1199] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:39:59.789330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575bb42-c8d8-414e-9335-eceab07aaaa4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:59.789728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575bb42-c8d8-414e-9335-eceab07aaaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:59.789865 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575bb42-c8d8-414e-9335-eceab07aaaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:59.809372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e575bb42-c8d8-414e-9335-eceab07aaaa4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:39:59.810591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e575bb42-c8d8-414e-9335-eceab07aaaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:39:59.819187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 601/1200] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:39:59.973638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff276a7-b1d0-4d67-a1bf-06c46c57c4ca None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:39:59.988733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff276a7-b1d0-4d67-a1bf-06c46c57c4ca None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:39:59.994118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b9a42-e92a-4280-bb1a-ba46d3c4834d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:39:59.994741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b9a42-e92a-4280-bb1a-ba46d3c4834d service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:59.995019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b9a42-e92a-4280-bb1a-ba46d3c4834d service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:59.995289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b9a42-e92a-4280-bb1a-ba46d3c4834d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:00.003892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b9a42-e92a-4280-bb1a-ba46d3c4834d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:40:00.005449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b9a42-e92a-4280-bb1a-ba46d3c4834d service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:40:00.014608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 600/1201] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:40:01.101074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30183cc-4057-40a2-99d4-c0cc02d4cbee None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:01.101681 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30183cc-4057-40a2-99d4-c0cc02d4cbee None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:01.101681 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30183cc-4057-40a2-99d4-c0cc02d4cbee None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:01.103035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 602/1202] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:40:01.107850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9302d2-13fd-4fde-9090-fff46de17986 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:01.108302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9302d2-13fd-4fde-9090-fff46de17986 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:01.108411 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9302d2-13fd-4fde-9090-fff46de17986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:01.128146 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-fb9302d2-13fd-4fde-9090-fff46de17986 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:40:01.129316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fb9302d2-13fd-4fde-9090-fff46de17986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:01.136919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 601/1203] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:40:01.289778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30183cc-4057-40a2-99d4-c0cc02d4cbee None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:40:01.304380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30183cc-4057-40a2-99d4-c0cc02d4cbee None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:40:01.309642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965ce9c-d5ae-4510-b00c-c63e72834191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:40:01.310274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965ce9c-d5ae-4510-b00c-c63e72834191 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:01.310551 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965ce9c-d5ae-4510-b00c-c63e72834191 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:01.310816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965ce9c-d5ae-4510-b00c-c63e72834191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:01.319418 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965ce9c-d5ae-4510-b00c-c63e72834191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:40:01.320958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965ce9c-d5ae-4510-b00c-c63e72834191 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:40:01.330107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 603/1204] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:40:05.432585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bd67f-9d8f-4e89-b78a-f76500899111 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:05.433142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bd67f-9d8f-4e89-b78a-f76500899111 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:05.433142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bd67f-9d8f-4e89-b78a-f76500899111 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:05.434609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 602/1205] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:40:05.440109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572462e-b551-40f0-a0ba-91f90fcff2a2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:05.440567 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572462e-b551-40f0-a0ba-91f90fcff2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:05.440674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572462e-b551-40f0-a0ba-91f90fcff2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:05.459911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a572462e-b551-40f0-a0ba-91f90fcff2a2 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:40:05.461003 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a572462e-b551-40f0-a0ba-91f90fcff2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:05.469681 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 604/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:40:05.757165 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bd67f-9d8f-4e89-b78a-f76500899111 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:40:05.758864 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-562bd67f-9d8f-4e89-b78a-f76500899111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:05.838227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bd67f-9d8f-4e89-b78a-f76500899111 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:40:05.841729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929175aa-ddfb-4483-afad-36e7de656e43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:40:05.842367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929175aa-ddfb-4483-afad-36e7de656e43 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:05.842651 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929175aa-ddfb-4483-afad-36e7de656e43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:05.842923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929175aa-ddfb-4483-afad-36e7de656e43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:05.852262 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929175aa-ddfb-4483-afad-36e7de656e43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:40:05.853680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929175aa-ddfb-4483-afad-36e7de656e43 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:40:05.862977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 603/1207] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:40:08.013841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964edfe-144f-4e8d-bd2e-2bcb26dac6bf None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:08.014460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964edfe-144f-4e8d-bd2e-2bcb26dac6bf None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:08.014460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964edfe-144f-4e8d-bd2e-2bcb26dac6bf None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:08.015806 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 605/1208] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:40:08.021388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4e8cd-145f-48ea-b2db-bc62b11aea5e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:08.021807 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4e8cd-145f-48ea-b2db-bc62b11aea5e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:08.021934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4e8cd-145f-48ea-b2db-bc62b11aea5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:08.041884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-def4e8cd-145f-48ea-b2db-bc62b11aea5e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:40:08.043229 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-def4e8cd-145f-48ea-b2db-bc62b11aea5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:08.051802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 604/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:40:08.340927 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964edfe-144f-4e8d-bd2e-2bcb26dac6bf None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:40:08.342710 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1964edfe-144f-4e8d-bd2e-2bcb26dac6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:08.355960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964edfe-144f-4e8d-bd2e-2bcb26dac6bf None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:40:08.360059 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafadd88-dbfe-45f7-827b-38a3f622bef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:40:08.360673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafadd88-dbfe-45f7-827b-38a3f622bef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:08.360928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafadd88-dbfe-45f7-827b-38a3f622bef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:08.361197 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafadd88-dbfe-45f7-827b-38a3f622bef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:08.369895 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafadd88-dbfe-45f7-827b-38a3f622bef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:40:08.371327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafadd88-dbfe-45f7-827b-38a3f622bef2 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:40:08.380561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 606/1210] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:40:12.042662 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2e462-87d6-47d1-bcd7-d147799224f9 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:12.043242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2e462-87d6-47d1-bcd7-d147799224f9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:12.043242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2e462-87d6-47d1-bcd7-d147799224f9 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:12.044592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 605/1211] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:40:12.050036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f1533-32b7-407a-be7f-8b3e2c360ea5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:12.050487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f1533-32b7-407a-be7f-8b3e2c360ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:12.050622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f1533-32b7-407a-be7f-8b3e2c360ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:12.058243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=81 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:40:12.071458 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f64f1533-32b7-407a-be7f-8b3e2c360ea5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:40:12.072640 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f64f1533-32b7-407a-be7f-8b3e2c360ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:12.080960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 607/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:40:12.382067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2e462-87d6-47d1-bcd7-d147799224f9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:40:12.383791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-54b2e462-87d6-47d1-bcd7-d147799224f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:12.448333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:40:12.461676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2e462-87d6-47d1-bcd7-d147799224f9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:40:12.465278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9a65f-7b52-4c55-aa30-b6fbaa277502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:40:12.465877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9a65f-7b52-4c55-aa30-b6fbaa277502 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:12.466158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9a65f-7b52-4c55-aa30-b6fbaa277502 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:12.466460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9a65f-7b52-4c55-aa30-b6fbaa277502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:12.475564 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9a65f-7b52-4c55-aa30-b6fbaa277502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:40:12.477009 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9a65f-7b52-4c55-aa30-b6fbaa277502 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:40:12.487123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 606/1213] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:40:14.647965 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe0087-9682-4eef-b217-14872c5ac22d None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:14.648508 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe0087-9682-4eef-b217-14872c5ac22d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:14.648508 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe0087-9682-4eef-b217-14872c5ac22d None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:14.649869 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 608/1214] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:40:14.655123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69eaa21-a559-4378-ad87-554b4090e42e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:14.655524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69eaa21-a559-4378-ad87-554b4090e42e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:14.655671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69eaa21-a559-4378-ad87-554b4090e42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:14.675661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f69eaa21-a559-4378-ad87-554b4090e42e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:40:14.676841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f69eaa21-a559-4378-ad87-554b4090e42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:14.684753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 607/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:40:14.982473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe0087-9682-4eef-b217-14872c5ac22d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:40:14.984157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3bbe0087-9682-4eef-b217-14872c5ac22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:14.997527 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe0087-9682-4eef-b217-14872c5ac22d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:40:15.001837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293950a-78b6-4ec0-8024-72f10830c8d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:40:15.002523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293950a-78b6-4ec0-8024-72f10830c8d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:15.002799 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293950a-78b6-4ec0-8024-72f10830c8d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:15.003068 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293950a-78b6-4ec0-8024-72f10830c8d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:15.011625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293950a-78b6-4ec0-8024-72f10830c8d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:40:15.013019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293950a-78b6-4ec0-8024-72f10830c8d7 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:40:15.022580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 609/1216] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:40:24.670272 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=11 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:40:25.018651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=8 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:38.702490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcc8b4-3a33-4f42-b2b3-cc18467ae461 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:38.704671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcc8b4-3a33-4f42-b2b3-cc18467ae461 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:38.704671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcc8b4-3a33-4f42-b2b3-cc18467ae461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:38.727284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-19fcc8b4-3a33-4f42-b2b3-cc18467ae461 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:38.728491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-19fcc8b4-3a33-4f42-b2b3-cc18467ae461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:38.741888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 608/1217] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:38.764363 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f471f7-22dc-4f87-9e71-7ed4451d4cbd None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:38.764960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f471f7-22dc-4f87-9e71-7ed4451d4cbd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:38.764960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f471f7-22dc-4f87-9e71-7ed4451d4cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:38.793166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e6f471f7-22dc-4f87-9e71-7ed4451d4cbd None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:38.797962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e6f471f7-22dc-4f87-9e71-7ed4451d4cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:38.810966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 610/1218] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:38.815715 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcc8b4-3a33-4f42-b2b3-cc18467ae461 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:38.827473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcc8b4-3a33-4f42-b2b3-cc18467ae461 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:38.838800 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de750d0e-0a8f-4ea6-8ba2-c7a11bd13a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:38.839840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de750d0e-0a8f-4ea6-8ba2-c7a11bd13a19 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:38.840333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de750d0e-0a8f-4ea6-8ba2-c7a11bd13a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:38.840821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de750d0e-0a8f-4ea6-8ba2-c7a11bd13a19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:38.841683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de750d0e-0a8f-4ea6-8ba2-c7a11bd13a19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:41:38.842212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de750d0e-0a8f-4ea6-8ba2-c7a11bd13a19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:38.844153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de750d0e-0a8f-4ea6-8ba2-c7a11bd13a19 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:38.860020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 609/1219] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:41:38.869436 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f471f7-22dc-4f87-9e71-7ed4451d4cbd None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:38.884368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f471f7-22dc-4f87-9e71-7ed4451d4cbd None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:38.897021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849ddb3-cbf0-46bb-ad0d-0507372a21ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:38.897664 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849ddb3-cbf0-46bb-ad0d-0507372a21ca admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:38.897942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849ddb3-cbf0-46bb-ad0d-0507372a21ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:38.898235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849ddb3-cbf0-46bb-ad0d-0507372a21ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:38.898959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849ddb3-cbf0-46bb-ad0d-0507372a21ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:38.901040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849ddb3-cbf0-46bb-ad0d-0507372a21ca admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:38.960543 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 611/1220] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:38 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:38.964670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de750d0e-0a8f-4ea6-8ba2-c7a11bd13a19 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:38.982570 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de750d0e-0a8f-4ea6-8ba2-c7a11bd13a19 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.076314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ebc43-8967-4020-9dd6-f321413cdfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.077342 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ebc43-8967-4020-9dd6-f321413cdfc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.077839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ebc43-8967-4020-9dd6-f321413cdfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.078352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ebc43-8967-4020-9dd6-f321413cdfc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.079340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ebc43-8967-4020-9dd6-f321413cdfc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.082437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ebc43-8967-4020-9dd6-f321413cdfc1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:39.128913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 610/1221] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:38 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:39.131686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849ddb3-cbf0-46bb-ad0d-0507372a21ca admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.145456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849ddb3-cbf0-46bb-ad0d-0507372a21ca admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.153603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358fe58-c519-46d4-8146-6839cb60beb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.154408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358fe58-c519-46d4-8146-6839cb60beb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.154684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358fe58-c519-46d4-8146-6839cb60beb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.154974 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358fe58-c519-46d4-8146-6839cb60beb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.155601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358fe58-c519-46d4-8146-6839cb60beb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.157432 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358fe58-c519-46d4-8146-6839cb60beb7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:39.172036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 612/1222] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:39.177410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ebc43-8967-4020-9dd6-f321413cdfc1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.193223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ebc43-8967-4020-9dd6-f321413cdfc1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.201546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9707d1-f033-4855-8572-c9f7a933bf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.202753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9707d1-f033-4855-8572-c9f7a933bf6d admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.203280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9707d1-f033-4855-8572-c9f7a933bf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.203771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9707d1-f033-4855-8572-c9f7a933bf6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.204743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9707d1-f033-4855-8572-c9f7a933bf6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.207789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9707d1-f033-4855-8572-c9f7a933bf6d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:39.229344 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 611/1223] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:39.233399 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358fe58-c519-46d4-8146-6839cb60beb7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.246113 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358fe58-c519-46d4-8146-6839cb60beb7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.259304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dab0f-b00a-4767-9ca1-9bebd95a82ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.260130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dab0f-b00a-4767-9ca1-9bebd95a82ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.260431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dab0f-b00a-4767-9ca1-9bebd95a82ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.260722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dab0f-b00a-4767-9ca1-9bebd95a82ab admin admin] PATH_INFO: `/v3/projects/e0f08e43a5984a9398b3d5ec41e4282c/users/1ea8feefb0614972898c1b1ad28077a8/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.285828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dab0f-b00a-4767-9ca1-9bebd95a82ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f08e43a5984a9398b3d5ec41e4282c, user_id=1ea8feefb0614972898c1b1ad28077a8, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.288919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dab0f-b00a-4767-9ca1-9bebd95a82ab admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:39.318843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 613/1224] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:39 2022] PUT /identity/v3/projects/e0f08e43a5984a9398b3d5ec41e4282c/users/1ea8feefb0614972898c1b1ad28077a8/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:39.322983 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492c441-9364-43db-b87d-d22f1b0b42f4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.323553 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492c441-9364-43db-b87d-d22f1b0b42f4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.324058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492c441-9364-43db-b87d-d22f1b0b42f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.331782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e70fb-11f7-410d-880f-69f542a60365 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.332098 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e70fb-11f7-410d-880f-69f542a60365 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.332329 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e70fb-11f7-410d-880f-69f542a60365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.365592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fb6e70fb-11f7-410d-880f-69f542a60365 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.366881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fb6e70fb-11f7-410d-880f-69f542a60365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.367281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c492c441-9364-43db-b87d-d22f1b0b42f4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.368914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c492c441-9364-43db-b87d-d22f1b0b42f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.489676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 614/1225] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:41:39.492523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c126b-be23-43d2-aa1e-bee89e5c26ee None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.492523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c126b-be23-43d2-aa1e-bee89e5c26ee None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.492693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c126b-be23-43d2-aa1e-bee89e5c26ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.522010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 612/1226] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.527569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf8e5c-e6e8-46ba-8029-ee4409635d9c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.528012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf8e5c-e6e8-46ba-8029-ee4409635d9c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.528293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf8e5c-e6e8-46ba-8029-ee4409635d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.543239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-248c126b-be23-43d2-aa1e-bee89e5c26ee None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.545282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-248c126b-be23-43d2-aa1e-bee89e5c26ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.555031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 615/1227] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.556284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c126b-be23-43d2-aa1e-bee89e5c26ee None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.556693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0cdf8e5c-e6e8-46ba-8029-ee4409635d9c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.558026 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0cdf8e5c-e6e8-46ba-8029-ee4409635d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.567851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 613/1228] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.570206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f522d-a60d-403a-a18a-20736954de69 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.570547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f522d-a60d-403a-a18a-20736954de69 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.570823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f522d-a60d-403a-a18a-20736954de69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.592459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-659f522d-a60d-403a-a18a-20736954de69 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.593649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-659f522d-a60d-403a-a18a-20736954de69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.603134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 614/1229] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.609877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f0c42-81c4-45b3-bff0-855b9a88009b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.609877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f0c42-81c4-45b3-bff0-855b9a88009b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.609877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f0c42-81c4-45b3-bff0-855b9a88009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.627247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c126b-be23-43d2-aa1e-bee89e5c26ee None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.631377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58202c31-990d-4dbb-9b6c-819c59b84bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.631530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d29f0c42-81c4-45b3-bff0-855b9a88009b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.631981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58202c31-990d-4dbb-9b6c-819c59b84bdf service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.632255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58202c31-990d-4dbb-9b6c-819c59b84bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.632521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58202c31-990d-4dbb-9b6c-819c59b84bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.632691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d29f0c42-81c4-45b3-bff0-855b9a88009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.640569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 615/1230] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.641828 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fd2ce-f472-4262-b8dd-3f0f86ea6b74 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.642062 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58202c31-990d-4dbb-9b6c-819c59b84bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.642301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fd2ce-f472-4262-b8dd-3f0f86ea6b74 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.642415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fd2ce-f472-4262-b8dd-3f0f86ea6b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.644414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58202c31-990d-4dbb-9b6c-819c59b84bdf service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:39.654683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 616/1231] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:39.655951 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858082b-03b5-4453-9eca-f1e4b8aacb33 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.656188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858082b-03b5-4453-9eca-f1e4b8aacb33 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.656460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858082b-03b5-4453-9eca-f1e4b8aacb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.664098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-795fd2ce-f472-4262-b8dd-3f0f86ea6b74 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.665305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-795fd2ce-f472-4262-b8dd-3f0f86ea6b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.674975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 616/1232] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.680281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f216538-4271-4341-9e4a-9c52dee946e5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.680569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f216538-4271-4341-9e4a-9c52dee946e5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.680841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f216538-4271-4341-9e4a-9c52dee946e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.694308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e858082b-03b5-4453-9eca-f1e4b8aacb33 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.695736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e858082b-03b5-4453-9eca-f1e4b8aacb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.702243 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0f216538-4271-4341-9e4a-9c52dee946e5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.703399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0f216538-4271-4341-9e4a-9c52dee946e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.705379 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 617/1233] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.708091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198f359-3aaa-47ca-a58e-8219360a6bbc None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.708420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198f359-3aaa-47ca-a58e-8219360a6bbc None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.708700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198f359-3aaa-47ca-a58e-8219360a6bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.717194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 617/1234] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.722079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5228c-2bc5-4148-bc67-43328ba952a6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.722407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5228c-2bc5-4148-bc67-43328ba952a6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.722628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5228c-2bc5-4148-bc67-43328ba952a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.730074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-6198f359-3aaa-47ca-a58e-8219360a6bbc None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.731398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6198f359-3aaa-47ca-a58e-8219360a6bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.746653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 618/1235] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.746653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc431e-2ed4-4500-a929-d1e46f403f8d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.746653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc431e-2ed4-4500-a929-d1e46f403f8d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.746653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc431e-2ed4-4500-a929-d1e46f403f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.747372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-5ff5228c-2bc5-4148-bc67-43328ba952a6 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.748681 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5ff5228c-2bc5-4148-bc67-43328ba952a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.759443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 618/1236] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.760355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5228c-2bc5-4148-bc67-43328ba952a6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.770560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-48dc431e-2ed4-4500-a929-d1e46f403f8d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.772381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-48dc431e-2ed4-4500-a929-d1e46f403f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.773502 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5228c-2bc5-4148-bc67-43328ba952a6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.778500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed69d9-5721-45f8-9243-93304f74ea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.779108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed69d9-5721-45f8-9243-93304f74ea6f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.779381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed69d9-5721-45f8-9243-93304f74ea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.779665 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed69d9-5721-45f8-9243-93304f74ea6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.780286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed69d9-5721-45f8-9243-93304f74ea6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:41:39.780580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed69d9-5721-45f8-9243-93304f74ea6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.781340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 619/1237] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.782189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed69d9-5721-45f8-9243-93304f74ea6f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:39.782604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bec693-54e9-4349-9a4a-cbacf06fe33b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.782977 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bec693-54e9-4349-9a4a-cbacf06fe33b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.783339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bec693-54e9-4349-9a4a-cbacf06fe33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.794473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:39.796274 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 619/1238] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:41:39.797310 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed69d9-5721-45f8-9243-93304f74ea6f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.807795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-67bec693-54e9-4349-9a4a-cbacf06fe33b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.809236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-67bec693-54e9-4349-9a4a-cbacf06fe33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.815877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed69d9-5721-45f8-9243-93304f74ea6f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.824284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 620/1239] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.824284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bec693-54e9-4349-9a4a-cbacf06fe33b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.825339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a82f2-27cb-4a69-8f0b-9dc117360410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.825965 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a82f2-27cb-4a69-8f0b-9dc117360410 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.826270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a82f2-27cb-4a69-8f0b-9dc117360410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.826541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a82f2-27cb-4a69-8f0b-9dc117360410 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.827562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a82f2-27cb-4a69-8f0b-9dc117360410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:41:39.827740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a82f2-27cb-4a69-8f0b-9dc117360410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.829347 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a82f2-27cb-4a69-8f0b-9dc117360410 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:39.831310 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:39.837086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bec693-54e9-4349-9a4a-cbacf06fe33b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.842331 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851c61-0abe-4ee7-89bb-cd1d8dacdee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.843021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851c61-0abe-4ee7-89bb-cd1d8dacdee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.843441 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851c61-0abe-4ee7-89bb-cd1d8dacdee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.843750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851c61-0abe-4ee7-89bb-cd1d8dacdee7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.844572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4851c61-0abe-4ee7-89bb-cd1d8dacdee7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:41:39.844882 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4851c61-0abe-4ee7-89bb-cd1d8dacdee7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.845923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-e4851c61-0abe-4ee7-89bb-cd1d8dacdee7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:41:39.845923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Oct 24 13:41:39.845923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: Oct 24 13:41:39.847193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4851c61-0abe-4ee7-89bb-cd1d8dacdee7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:39.847485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 620/1240] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:41:39.849276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d931226-b237-4247-aba1-2bf16a871efa None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.849970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d931226-b237-4247-aba1-2bf16a871efa None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.850382 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d931226-b237-4247-aba1-2bf16a871efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.869345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 621/1241] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:41:39.869345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851c61-0abe-4ee7-89bb-cd1d8dacdee7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.874639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4d931226-b237-4247-aba1-2bf16a871efa None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.874639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4d931226-b237-4247-aba1-2bf16a871efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.882437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 621/1242] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.883771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a2ad7-18a9-4dc1-867c-9d66e6e7f0f4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.884053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a2ad7-18a9-4dc1-867c-9d66e6e7f0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.884332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a2ad7-18a9-4dc1-867c-9d66e6e7f0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.896698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851c61-0abe-4ee7-89bb-cd1d8dacdee7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.901511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce06f1-3fbb-49e0-9535-8ca4eab0dc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.902693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce06f1-3fbb-49e0-9535-8ca4eab0dc26 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.903060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce06f1-3fbb-49e0-9535-8ca4eab0dc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.903342 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce06f1-3fbb-49e0-9535-8ca4eab0dc26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.904864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce06f1-3fbb-49e0-9535-8ca4eab0dc26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:41:39.905184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce06f1-3fbb-49e0-9535-8ca4eab0dc26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.907011 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce06f1-3fbb-49e0-9535-8ca4eab0dc26 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:39.907590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-5f5a2ad7-18a9-4dc1-867c-9d66e6e7f0f4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:39.909408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5f5a2ad7-18a9-4dc1-867c-9d66e6e7f0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:39.919686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 622/1243] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:39.920495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a2ad7-18a9-4dc1-867c-9d66e6e7f0f4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.921077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 622/1244] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:41:39.921804 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce06f1-3fbb-49e0-9535-8ca4eab0dc26 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.933836 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce06f1-3fbb-49e0-9535-8ca4eab0dc26 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.934129 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a2ad7-18a9-4dc1-867c-9d66e6e7f0f4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.938427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb5a95-8472-4c28-a47d-867b4d1e3b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.938840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2ec40-d9fc-4422-9795-989ced900247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.939034 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb5a95-8472-4c28-a47d-867b4d1e3b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.939302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb5a95-8472-4c28-a47d-867b4d1e3b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.939425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2ec40-d9fc-4422-9795-989ced900247 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.939616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb5a95-8472-4c28-a47d-867b4d1e3b20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.939743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2ec40-d9fc-4422-9795-989ced900247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.940558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2ec40-d9fc-4422-9795-989ced900247 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.940991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb5a95-8472-4c28-a47d-867b4d1e3b20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:41:39.941510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb5a95-8472-4c28-a47d-867b4d1e3b20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.941702 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2ec40-d9fc-4422-9795-989ced900247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.942794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-75c2ec40-d9fc-4422-9795-989ced900247 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:41:39.942794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Oct 24 13:41:39.942794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: Oct 24 13:41:39.943253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb5a95-8472-4c28-a47d-867b4d1e3b20 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:39.944029 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2ec40-d9fc-4422-9795-989ced900247 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:39.956962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 623/1245] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:41:39.958756 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb5a95-8472-4c28-a47d-867b4d1e3b20 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.974088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb5a95-8472-4c28-a47d-867b4d1e3b20 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:39.980254 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b293a-c0af-4331-adae-84cfcceacc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:39.980877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b293a-c0af-4331-adae-84cfcceacc7f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:39.981156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b293a-c0af-4331-adae-84cfcceacc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:39.981432 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b293a-c0af-4331-adae-84cfcceacc7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:39.982362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b293a-c0af-4331-adae-84cfcceacc7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:39.984248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 623/1246] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:39.984999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2ec40-d9fc-4422-9795-989ced900247 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:39.988137 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b293a-c0af-4331-adae-84cfcceacc7f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.030519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 624/1247] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.030519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b293a-c0af-4331-adae-84cfcceacc7f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.072025 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2ec40-d9fc-4422-9795-989ced900247 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.134144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b293a-c0af-4331-adae-84cfcceacc7f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.136328 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b3ff6-690a-48bb-a564-21d1c6ea9e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.136947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b3ff6-690a-48bb-a564-21d1c6ea9e97 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.137291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b3ff6-690a-48bb-a564-21d1c6ea9e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.137570 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b3ff6-690a-48bb-a564-21d1c6ea9e97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.138262 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b3ff6-690a-48bb-a564-21d1c6ea9e97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.140184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8e400-a59f-4c11-b63d-7c593242e8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.140339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b3ff6-690a-48bb-a564-21d1c6ea9e97 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.140524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e400-a59f-4c11-b63d-7c593242e8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.140805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e400-a59f-4c11-b63d-7c593242e8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.141081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8e400-a59f-4c11-b63d-7c593242e8b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.141735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8e400-a59f-4c11-b63d-7c593242e8b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:41:40.142108 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8e400-a59f-4c11-b63d-7c593242e8b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.143891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8e400-a59f-4c11-b63d-7c593242e8b5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.156872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 625/1248] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:41:40.163270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8e400-a59f-4c11-b63d-7c593242e8b5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.176745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8e400-a59f-4c11-b63d-7c593242e8b5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.177897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 624/1249] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:39 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.178789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b3ff6-690a-48bb-a564-21d1c6ea9e97 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.236664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:40.246994 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663974d7-bb5c-4189-8333-0563feae9b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.247650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663974d7-bb5c-4189-8333-0563feae9b1a admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.247913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663974d7-bb5c-4189-8333-0563feae9b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.248180 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663974d7-bb5c-4189-8333-0563feae9b1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.248836 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663974d7-bb5c-4189-8333-0563feae9b1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:41:40.249137 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663974d7-bb5c-4189-8333-0563feae9b1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.250995 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663974d7-bb5c-4189-8333-0563feae9b1a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.252641 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b3ff6-690a-48bb-a564-21d1c6ea9e97 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.258681 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a1f3a-b5df-4395-ae90-1e1f60282fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.260206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a1f3a-b5df-4395-ae90-1e1f60282fab admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.260790 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a1f3a-b5df-4395-ae90-1e1f60282fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.261710 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a1f3a-b5df-4395-ae90-1e1f60282fab admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.262763 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a1f3a-b5df-4395-ae90-1e1f60282fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.264838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a1f3a-b5df-4395-ae90-1e1f60282fab admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.272819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 626/1250] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:41:40.272819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663974d7-bb5c-4189-8333-0563feae9b1a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.278934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663974d7-bb5c-4189-8333-0563feae9b1a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.284961 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f85ca6-2da0-45f9-b5dd-130e45317d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.285735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f85ca6-2da0-45f9-b5dd-130e45317d14 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.286132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f85ca6-2da0-45f9-b5dd-130e45317d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.286539 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f85ca6-2da0-45f9-b5dd-130e45317d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.287280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f85ca6-2da0-45f9-b5dd-130e45317d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.289571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f85ca6-2da0-45f9-b5dd-130e45317d14 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.302573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 625/1251] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:40 2022] POST /identity/v3/projects => generated 357 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.303414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a1f3a-b5df-4395-ae90-1e1f60282fab admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.326918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 627/1252] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:40 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.328003 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f85ca6-2da0-45f9-b5dd-130e45317d14 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.333120 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:40.385771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a1f3a-b5df-4395-ae90-1e1f60282fab admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.398010 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f85ca6-2da0-45f9-b5dd-130e45317d14 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.403059 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92734dda-6c5c-4d81-b423-a8553067e25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.403834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92734dda-6c5c-4d81-b423-a8553067e25d admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.404178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92734dda-6c5c-4d81-b423-a8553067e25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.404522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92734dda-6c5c-4d81-b423-a8553067e25d admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.405241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92734dda-6c5c-4d81-b423-a8553067e25d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.406358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-92734dda-6c5c-4d81-b423-a8553067e25d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:41:40.406358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Oct 24 13:41:40.406358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: Oct 24 13:41:40.407682 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92734dda-6c5c-4d81-b423-a8553067e25d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.443689 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 628/1253] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:40 2022] POST /identity/v3/users => generated 303 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.443689 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92734dda-6c5c-4d81-b423-a8553067e25d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.451476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92734dda-6c5c-4d81-b423-a8553067e25d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.456848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9581e0a-d4ab-4396-9b14-04e802ed7ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.457452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9581e0a-d4ab-4396-9b14-04e802ed7ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.457731 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9581e0a-d4ab-4396-9b14-04e802ed7ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.458106 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9581e0a-d4ab-4396-9b14-04e802ed7ee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.458671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9581e0a-d4ab-4396-9b14-04e802ed7ee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.460573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9581e0a-d4ab-4396-9b14-04e802ed7ee8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.464839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace7917-e81f-4a49-a025-2561e8f7e7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.465626 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace7917-e81f-4a49-a025-2561e8f7e7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.465897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace7917-e81f-4a49-a025-2561e8f7e7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.466184 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace7917-e81f-4a49-a025-2561e8f7e7a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.466863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace7917-e81f-4a49-a025-2561e8f7e7a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.468804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace7917-e81f-4a49-a025-2561e8f7e7a6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.500993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 626/1254] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:40 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.501829 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace7917-e81f-4a49-a025-2561e8f7e7a6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.505897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 629/1255] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:40 2022] POST /identity/v3/projects => generated 393 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.509963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9581e0a-d4ab-4396-9b14-04e802ed7ee8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.513196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace7917-e81f-4a49-a025-2561e8f7e7a6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.600724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c194d-8158-4bc4-adac-0f922bdd62e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.601472 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c194d-8158-4bc4-adac-0f922bdd62e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.601863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c194d-8158-4bc4-adac-0f922bdd62e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.602266 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c194d-8158-4bc4-adac-0f922bdd62e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.602521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9581e0a-d4ab-4396-9b14-04e802ed7ee8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.603229 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c194d-8158-4bc4-adac-0f922bdd62e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.605761 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c194d-8158-4bc4-adac-0f922bdd62e5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.607372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554fd79-1b76-404c-a5e7-3e030f2b89a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.608088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554fd79-1b76-404c-a5e7-3e030f2b89a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.608425 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554fd79-1b76-404c-a5e7-3e030f2b89a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.608699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554fd79-1b76-404c-a5e7-3e030f2b89a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.609341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554fd79-1b76-404c-a5e7-3e030f2b89a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.611733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554fd79-1b76-404c-a5e7-3e030f2b89a5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.635314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 627/1256] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:40 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.635314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c194d-8158-4bc4-adac-0f922bdd62e5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.639003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:40.646833 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c194d-8158-4bc4-adac-0f922bdd62e5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.652588 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 630/1257] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:40 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.654491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095510e-0a85-4251-b2aa-20914371747d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.655660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095510e-0a85-4251-b2aa-20914371747d admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.656367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095510e-0a85-4251-b2aa-20914371747d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.656820 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095510e-0a85-4251-b2aa-20914371747d admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.657153 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554fd79-1b76-404c-a5e7-3e030f2b89a5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.657805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095510e-0a85-4251-b2aa-20914371747d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.661453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095510e-0a85-4251-b2aa-20914371747d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.689342 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 628/1258] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:40 2022] POST /identity/v3/users => generated 279 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.690572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095510e-0a85-4251-b2aa-20914371747d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.702845 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095510e-0a85-4251-b2aa-20914371747d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.731958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:40.739092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554fd79-1b76-404c-a5e7-3e030f2b89a5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.747063 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f7ad3-815d-4db8-91e2-3a973fca5e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.747841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f7ad3-815d-4db8-91e2-3a973fca5e4f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.748255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f7ad3-815d-4db8-91e2-3a973fca5e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.748668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f7ad3-815d-4db8-91e2-3a973fca5e4f admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.749470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f7ad3-815d-4db8-91e2-3a973fca5e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.752795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f7ad3-815d-4db8-91e2-3a973fca5e4f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.784973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 631/1259] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:40 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.784973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f7ad3-815d-4db8-91e2-3a973fca5e4f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.792952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f0f93-0efc-4242-a250-0db1e4144a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.793858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f0f93-0efc-4242-a250-0db1e4144a53 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.794368 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f0f93-0efc-4242-a250-0db1e4144a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.794884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f0f93-0efc-4242-a250-0db1e4144a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.795450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f7ad3-815d-4db8-91e2-3a973fca5e4f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.795784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f0f93-0efc-4242-a250-0db1e4144a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.797664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f0f93-0efc-4242-a250-0db1e4144a53 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.803062 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abc142-b322-4052-a614-5340605a598b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.803784 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abc142-b322-4052-a614-5340605a598b admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.804133 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abc142-b322-4052-a614-5340605a598b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.804477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abc142-b322-4052-a614-5340605a598b admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.805196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abc142-b322-4052-a614-5340605a598b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.808127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abc142-b322-4052-a614-5340605a598b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.814549 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 629/1260] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:40.816754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f0f93-0efc-4242-a250-0db1e4144a53 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.821863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 632/1261] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:40.822818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abc142-b322-4052-a614-5340605a598b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.832769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f0f93-0efc-4242-a250-0db1e4144a53 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.835048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abc142-b322-4052-a614-5340605a598b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.838343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec2f57-e86e-4935-8e5e-627997ac918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.839137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2f57-e86e-4935-8e5e-627997ac918e admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.839421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2f57-e86e-4935-8e5e-627997ac918e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.839801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec2f57-e86e-4935-8e5e-627997ac918e admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.840498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec2f57-e86e-4935-8e5e-627997ac918e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.840626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fe8d5-635d-40b8-9188-06fee15ab4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.841255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe8d5-635d-40b8-9188-06fee15ab4ef admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.841603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe8d5-635d-40b8-9188-06fee15ab4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.841935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fe8d5-635d-40b8-9188-06fee15ab4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.842795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fe8d5-635d-40b8-9188-06fee15ab4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.842911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec2f57-e86e-4935-8e5e-627997ac918e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.844502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fe8d5-635d-40b8-9188-06fee15ab4ef admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.858715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 633/1262] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:40.860184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fe8d5-635d-40b8-9188-06fee15ab4ef admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.874124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fe8d5-635d-40b8-9188-06fee15ab4ef admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.881063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 630/1263] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:40 2022] POST /identity/v3/users => generated 297 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.881484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e159f8-af2e-476f-836d-5ad8bb5863a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.882122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec2f57-e86e-4935-8e5e-627997ac918e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.882454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e159f8-af2e-476f-836d-5ad8bb5863a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.882879 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e159f8-af2e-476f-836d-5ad8bb5863a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.883364 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e159f8-af2e-476f-836d-5ad8bb5863a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.884122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e159f8-af2e-476f-836d-5ad8bb5863a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.887390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e159f8-af2e-476f-836d-5ad8bb5863a0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.893994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec2f57-e86e-4935-8e5e-627997ac918e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.899340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994b591-da56-45f3-b3a5-4eb8fed866e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.899955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994b591-da56-45f3-b3a5-4eb8fed866e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.900350 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994b591-da56-45f3-b3a5-4eb8fed866e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.900629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994b591-da56-45f3-b3a5-4eb8fed866e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.901294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994b591-da56-45f3-b3a5-4eb8fed866e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.903028 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994b591-da56-45f3-b3a5-4eb8fed866e3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.917527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 631/1264] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:40.918495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994b591-da56-45f3-b3a5-4eb8fed866e3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.920861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 634/1265] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:40 2022] POST /identity/v3/users => generated 292 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:40.921815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e159f8-af2e-476f-836d-5ad8bb5863a0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.933543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994b591-da56-45f3-b3a5-4eb8fed866e3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.934966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e159f8-af2e-476f-836d-5ad8bb5863a0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.938642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0e680-884a-4964-a1fa-f9926820d873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.939553 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0e680-884a-4964-a1fa-f9926820d873 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.940460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0e680-884a-4964-a1fa-f9926820d873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.940883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0e680-884a-4964-a1fa-f9926820d873 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.941836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0e680-884a-4964-a1fa-f9926820d873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.942929 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627576d-547b-4705-9617-4d8093b91962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.943611 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627576d-547b-4705-9617-4d8093b91962 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.943864 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0e680-884a-4964-a1fa-f9926820d873 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.943995 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627576d-547b-4705-9617-4d8093b91962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.944320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627576d-547b-4705-9617-4d8093b91962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.945081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627576d-547b-4705-9617-4d8093b91962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.946854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627576d-547b-4705-9617-4d8093b91962 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.956573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 632/1266] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:40.958516 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0e680-884a-4964-a1fa-f9926820d873 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.971672 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 635/1267] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:40.972664 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627576d-547b-4705-9617-4d8093b91962 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:40.975406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0e680-884a-4964-a1fa-f9926820d873 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.982622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadca93-91d4-42cd-aa81-609e778a75ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.983289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadca93-91d4-42cd-aa81-609e778a75ed admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.983553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627576d-547b-4705-9617-4d8093b91962 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:40.983684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadca93-91d4-42cd-aa81-609e778a75ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.984481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadca93-91d4-42cd-aa81-609e778a75ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.985214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadca93-91d4-42cd-aa81-609e778a75ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:40.988660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadca93-91d4-42cd-aa81-609e778a75ed admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:40.992995 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696da76a-196f-48a6-bbf9-9903fbaad79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:40.993614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696da76a-196f-48a6-bbf9-9903fbaad79e admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:40.993890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696da76a-196f-48a6-bbf9-9903fbaad79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:40.994158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696da76a-196f-48a6-bbf9-9903fbaad79e admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:40.998288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696da76a-196f-48a6-bbf9-9903fbaad79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.002791 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696da76a-196f-48a6-bbf9-9903fbaad79e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.007367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 633/1268] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:41.007935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadca93-91d4-42cd-aa81-609e778a75ed admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.018224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 636/1269] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:41.019845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696da76a-196f-48a6-bbf9-9903fbaad79e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.021681 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadca93-91d4-42cd-aa81-609e778a75ed admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.027759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a98b6-67ca-4a6d-8eca-66a7b5d9c647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.028584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a98b6-67ca-4a6d-8eca-66a7b5d9c647 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.030059 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a98b6-67ca-4a6d-8eca-66a7b5d9c647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.030322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a98b6-67ca-4a6d-8eca-66a7b5d9c647 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.031421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a98b6-67ca-4a6d-8eca-66a7b5d9c647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.033025 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a98b6-67ca-4a6d-8eca-66a7b5d9c647 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.037155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696da76a-196f-48a6-bbf9-9903fbaad79e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.044077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d025eb4-7596-43a1-b41d-73324b90520c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.044795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d025eb4-7596-43a1-b41d-73324b90520c admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.045071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d025eb4-7596-43a1-b41d-73324b90520c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.045415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d025eb4-7596-43a1-b41d-73324b90520c admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.046159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d025eb4-7596-43a1-b41d-73324b90520c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.047979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d025eb4-7596-43a1-b41d-73324b90520c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.053696 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 634/1270] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:41.055872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a98b6-67ca-4a6d-8eca-66a7b5d9c647 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.063137 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 637/1271] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:41.064226 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d025eb4-7596-43a1-b41d-73324b90520c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.075298 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a98b6-67ca-4a6d-8eca-66a7b5d9c647 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.075485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d025eb4-7596-43a1-b41d-73324b90520c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.080857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6699fb-c78f-4955-b5cb-717f37616142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.081582 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6699fb-c78f-4955-b5cb-717f37616142 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.081957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6699fb-c78f-4955-b5cb-717f37616142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.082284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6699fb-c78f-4955-b5cb-717f37616142 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.082663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad9e1a-3053-4253-bc77-fa60a36032c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.082968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6699fb-c78f-4955-b5cb-717f37616142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.083800 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad9e1a-3053-4253-bc77-fa60a36032c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.084247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad9e1a-3053-4253-bc77-fa60a36032c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.085409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6699fb-c78f-4955-b5cb-717f37616142 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.085685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad9e1a-3053-4253-bc77-fa60a36032c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.086578 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad9e1a-3053-4253-bc77-fa60a36032c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.088684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad9e1a-3053-4253-bc77-fa60a36032c4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.102377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 638/1272] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:41.103363 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6699fb-c78f-4955-b5cb-717f37616142 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.106824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 635/1273] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:41.107501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad9e1a-3053-4253-bc77-fa60a36032c4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.117195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6699fb-c78f-4955-b5cb-717f37616142 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.125467 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e8f55-2bb0-4abf-83e5-1a4dc8a33397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.125668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad9e1a-3053-4253-bc77-fa60a36032c4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.125759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e8f55-2bb0-4abf-83e5-1a4dc8a33397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.125759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e8f55-2bb0-4abf-83e5-1a4dc8a33397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.125946 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e8f55-2bb0-4abf-83e5-1a4dc8a33397 admin admin] PATH_INFO: `/v3/projects/fabcd819e2e941118fc937524f00d5ac/users/7c7c1846407a45aa812ff0e59ba8797b/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.132818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea529d4-76ca-4f0c-8264-01bdba21be25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.133623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea529d4-76ca-4f0c-8264-01bdba21be25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.133900 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea529d4-76ca-4f0c-8264-01bdba21be25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.134190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea529d4-76ca-4f0c-8264-01bdba21be25 admin admin] PATH_INFO: `/v3/projects/ab3a235b26aa4c93b2354d2af757873e/users/9cfcee4879df4843867f69c5091df370/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.149313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e8f55-2bb0-4abf-83e5-1a4dc8a33397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabcd819e2e941118fc937524f00d5ac, user_id=7c7c1846407a45aa812ff0e59ba8797b, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.152892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e8f55-2bb0-4abf-83e5-1a4dc8a33397 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.158286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea529d4-76ca-4f0c-8264-01bdba21be25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a235b26aa4c93b2354d2af757873e, user_id=9cfcee4879df4843867f69c5091df370, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.162318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea529d4-76ca-4f0c-8264-01bdba21be25 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.171419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 639/1274] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:41 2022] PUT /identity/v3/projects/fabcd819e2e941118fc937524f00d5ac/users/7c7c1846407a45aa812ff0e59ba8797b/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:41.172365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e8f55-2bb0-4abf-83e5-1a4dc8a33397 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.191687 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 636/1275] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:41 2022] PUT /identity/v3/projects/ab3a235b26aa4c93b2354d2af757873e/users/9cfcee4879df4843867f69c5091df370/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:41.195538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea529d4-76ca-4f0c-8264-01bdba21be25 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.251848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:41.275022 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea529d4-76ca-4f0c-8264-01bdba21be25 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.281249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9a5a9-9bb6-45fb-825f-90b14e1f4e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.281417 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e8f55-2bb0-4abf-83e5-1a4dc8a33397 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.282447 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9a5a9-9bb6-45fb-825f-90b14e1f4e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.282817 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9a5a9-9bb6-45fb-825f-90b14e1f4e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.283135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9a5a9-9bb6-45fb-825f-90b14e1f4e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.285205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9a5a9-9bb6-45fb-825f-90b14e1f4e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.287313 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9a5a9-9bb6-45fb-825f-90b14e1f4e79 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.287604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b199ff4-099e-4869-93f4-aab36aec53a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.288436 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199ff4-099e-4869-93f4-aab36aec53a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.288879 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199ff4-099e-4869-93f4-aab36aec53a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.289373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199ff4-099e-4869-93f4-aab36aec53a2 admin admin] PATH_INFO: `/v3/projects/fb21fa4a8de149b29f5324dc2d80464a/users/59988220318042cab2f81f9ed3a57caa/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.303506 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 637/1276] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:41.304363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9a5a9-9bb6-45fb-825f-90b14e1f4e79 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.314473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b199ff4-099e-4869-93f4-aab36aec53a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb21fa4a8de149b29f5324dc2d80464a, user_id=59988220318042cab2f81f9ed3a57caa, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.317537 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9a5a9-9bb6-45fb-825f-90b14e1f4e79 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.318509 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b199ff4-099e-4869-93f4-aab36aec53a2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.322634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d466e-a151-4061-bc30-3691959b803d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.323360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d466e-a151-4061-bc30-3691959b803d admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.323670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d466e-a151-4061-bc30-3691959b803d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.324003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d466e-a151-4061-bc30-3691959b803d admin admin] PATH_INFO: `/v3/projects/96dac3b390514fabb62cb2d86ef16d97/users/1bcd5392ff7a49908d9140fefea19ca0/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.337904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 640/1277] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:41 2022] PUT /identity/v3/projects/fb21fa4a8de149b29f5324dc2d80464a/users/59988220318042cab2f81f9ed3a57caa/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:41.339522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b199ff4-099e-4869-93f4-aab36aec53a2 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.341608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d466e-a151-4061-bc30-3691959b803d admin admin] RBAC: Authorizing `identity:create_grant(project_id=96dac3b390514fabb62cb2d86ef16d97, user_id=1bcd5392ff7a49908d9140fefea19ca0, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.344360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d466e-a151-4061-bc30-3691959b803d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.368777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 638/1278] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:41 2022] PUT /identity/v3/projects/96dac3b390514fabb62cb2d86ef16d97/users/1bcd5392ff7a49908d9140fefea19ca0/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:41.369532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d466e-a151-4061-bc30-3691959b803d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.383299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:41.443329 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b199ff4-099e-4869-93f4-aab36aec53a2 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.470975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d466e-a151-4061-bc30-3691959b803d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.482566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497699e9-8599-4bf1-abb5-bbad8f6282ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.483361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497699e9-8599-4bf1-abb5-bbad8f6282ec admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.484447 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497699e9-8599-4bf1-abb5-bbad8f6282ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.486271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497699e9-8599-4bf1-abb5-bbad8f6282ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.487725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497699e9-8599-4bf1-abb5-bbad8f6282ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.495013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497699e9-8599-4bf1-abb5-bbad8f6282ec admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.508928 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 639/1279] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:41.510014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249c003-7bbe-4180-86fe-e2d386f9e3e4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.510381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249c003-7bbe-4180-86fe-e2d386f9e3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.510660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249c003-7bbe-4180-86fe-e2d386f9e3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.520623 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39af5b1-efa3-4240-8734-10c9984ef78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.521297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af5b1-efa3-4240-8734-10c9984ef78e admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.521586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af5b1-efa3-4240-8734-10c9984ef78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.521860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39af5b1-efa3-4240-8734-10c9984ef78e admin admin] PATH_INFO: `/v3/projects/405f8a6f56404e62bbc875bf8f0b2209/users/4d877c3fa7524181ae29b4c75b51f330/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.541057 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39af5b1-efa3-4240-8734-10c9984ef78e admin admin] RBAC: Authorizing `identity:create_grant(project_id=405f8a6f56404e62bbc875bf8f0b2209, user_id=4d877c3fa7524181ae29b4c75b51f330, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.543692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39af5b1-efa3-4240-8734-10c9984ef78e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.545916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-5249c003-7bbe-4180-86fe-e2d386f9e3e4 None None] MFA Rules not processed for user `7c7c1846407a45aa812ff0e59ba8797b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:41.547173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5249c003-7bbe-4180-86fe-e2d386f9e3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:41.559845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 641/1280] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:41 2022] PUT /identity/v3/projects/405f8a6f56404e62bbc875bf8f0b2209/users/4d877c3fa7524181ae29b4c75b51f330/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:41.573288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e678c-fb59-462b-9be8-11ca17f4d11a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.573782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e678c-fb59-462b-9be8-11ca17f4d11a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.575583 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e678c-fb59-462b-9be8-11ca17f4d11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.598831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e28e678c-fb59-462b-9be8-11ca17f4d11a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:41.600055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e28e678c-fb59-462b-9be8-11ca17f4d11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:41.662023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 640/1281] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:41 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:41.663134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249c003-7bbe-4180-86fe-e2d386f9e3e4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.677679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249c003-7bbe-4180-86fe-e2d386f9e3e4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.707641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 642/1282] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:41.709349 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2310b-af17-4efc-b742-ea9094fa4bb9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.709894 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2310b-af17-4efc-b742-ea9094fa4bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.710489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2310b-af17-4efc-b742-ea9094fa4bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.743483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7df2310b-af17-4efc-b742-ea9094fa4bb9 None None] MFA Rules not processed for user `59988220318042cab2f81f9ed3a57caa`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:41.745246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7df2310b-af17-4efc-b742-ea9094fa4bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:41.748533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:41.776915 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d609808-ac23-450d-b5a7-0c0603bf9d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.777594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d609808-ac23-450d-b5a7-0c0603bf9d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.777941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d609808-ac23-450d-b5a7-0c0603bf9d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.778249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d609808-ac23-450d-b5a7-0c0603bf9d82 admin admin] PATH_INFO: `/v3/projects/d506ce8ca200496cbc15d09f49eab942/users/be793f4685f94beba74adedbc02ee50a/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.798998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d609808-ac23-450d-b5a7-0c0603bf9d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d506ce8ca200496cbc15d09f49eab942, user_id=be793f4685f94beba74adedbc02ee50a, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.802533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d609808-ac23-450d-b5a7-0c0603bf9d82 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.806377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:41.821114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 641/1283] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:41 2022] PUT /identity/v3/projects/d506ce8ca200496cbc15d09f49eab942/users/be793f4685f94beba74adedbc02ee50a/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:41.822841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6c0c3-49a5-4bd0-b61f-6d305edc410e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.823337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6c0c3-49a5-4bd0-b61f-6d305edc410e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.823780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6c0c3-49a5-4bd0-b61f-6d305edc410e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.847877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 643/1284] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:41.848372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2310b-af17-4efc-b742-ea9094fa4bb9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.856244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2cc6c0c3-49a5-4bd0-b61f-6d305edc410e None None] MFA Rules not processed for user `1bcd5392ff7a49908d9140fefea19ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:41.857696 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2cc6c0c3-49a5-4bd0-b61f-6d305edc410e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:41.859520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2310b-af17-4efc-b742-ea9094fa4bb9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:41.934881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36000557-35f7-4e84-98a8-910dd8854951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:41.935676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36000557-35f7-4e84-98a8-910dd8854951 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.936026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36000557-35f7-4e84-98a8-910dd8854951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.936373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36000557-35f7-4e84-98a8-910dd8854951 admin admin] PATH_INFO: `/v3/projects/ae58cf3c8178463296c8b633550a130e/users/1461a94dd8ef48f79a0d8642a1bc1e58/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.952335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 642/1285] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:41.954068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd3337-7e86-4ed9-94eb-e441a833ab24 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:41.954068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd3337-7e86-4ed9-94eb-e441a833ab24 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:41.954300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd3337-7e86-4ed9-94eb-e441a833ab24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:41.957362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36000557-35f7-4e84-98a8-910dd8854951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae58cf3c8178463296c8b633550a130e, user_id=1461a94dd8ef48f79a0d8642a1bc1e58, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:41.961060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36000557-35f7-4e84-98a8-910dd8854951 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:41.983032 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 644/1286] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:41 2022] PUT /identity/v3/projects/ae58cf3c8178463296c8b633550a130e/users/1461a94dd8ef48f79a0d8642a1bc1e58/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:41.983665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36000557-35f7-4e84-98a8-910dd8854951 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:41.988699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-22dd3337-7e86-4ed9-94eb-e441a833ab24 None None] MFA Rules not processed for user `4d877c3fa7524181ae29b4c75b51f330`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:41.989960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-22dd3337-7e86-4ed9-94eb-e441a833ab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:42.054643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36000557-35f7-4e84-98a8-910dd8854951 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:42.061181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959f25b-7bc1-4ba5-a49c-bae567e08063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:42.062537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959f25b-7bc1-4ba5-a49c-bae567e08063 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:42.062537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959f25b-7bc1-4ba5-a49c-bae567e08063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:42.062942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959f25b-7bc1-4ba5-a49c-bae567e08063 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:42.064070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959f25b-7bc1-4ba5-a49c-bae567e08063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:42.067322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959f25b-7bc1-4ba5-a49c-bae567e08063 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:42.070807 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 643/1287] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:42.075580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd3337-7e86-4ed9-94eb-e441a833ab24 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:42.111486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 645/1288] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:41 2022] POST /identity/v3/projects => generated 405 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:42.111486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42500d8-f9ed-4936-b277-8be2e70d48aa None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:42.112243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42500d8-f9ed-4936-b277-8be2e70d48aa None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:42.112843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42500d8-f9ed-4936-b277-8be2e70d48aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:42.142050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:42.155327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd3337-7e86-4ed9-94eb-e441a833ab24 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:42.156126 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c42500d8-f9ed-4936-b277-8be2e70d48aa None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:42.161094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c42500d8-f9ed-4936-b277-8be2e70d48aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:42.240842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c50e1-f0bf-4707-8911-733ac9d52950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:42.243000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c50e1-f0bf-4707-8911-733ac9d52950 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:42.243285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c50e1-f0bf-4707-8911-733ac9d52950 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:42.243560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c50e1-f0bf-4707-8911-733ac9d52950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:42.263552 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:42.310132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 646/1289] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:42.312474 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a9a21-09f4-4c33-b03b-4c468ac725a9 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:42.312858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a9a21-09f4-4c33-b03b-4c468ac725a9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:42.313279 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a9a21-09f4-4c33-b03b-4c468ac725a9 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:42.316845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 647/1290] 10.222.0.20 () {54 vars in 961 bytes} [Mon Oct 24 13:41:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:41:42.323463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a9a21-09f4-4c33-b03b-4c468ac725a9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:42.350466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c50e1-f0bf-4707-8911-733ac9d52950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:42.352190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c50e1-f0bf-4707-8911-733ac9d52950 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:42.372543 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a9a21-09f4-4c33-b03b-4c468ac725a9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:42.379840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 644/1291] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:42.380774 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11ba77-a919-4c1b-aa58-c6649dc6928d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:42.381065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11ba77-a919-4c1b-aa58-c6649dc6928d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:42.384079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11ba77-a919-4c1b-aa58-c6649dc6928d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:42.385723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bcfde5-1b77-4932-8299-141c5875218d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:42.386488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcfde5-1b77-4932-8299-141c5875218d admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:42.386944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcfde5-1b77-4932-8299-141c5875218d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:42.387306 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcfde5-1b77-4932-8299-141c5875218d admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:42.388041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bcfde5-1b77-4932-8299-141c5875218d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:42.390047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bcfde5-1b77-4932-8299-141c5875218d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:42.422726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bd11ba77-a919-4c1b-aa58-c6649dc6928d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:42.424010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bd11ba77-a919-4c1b-aa58-c6649dc6928d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:42.449746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 645/1292] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:42.454726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11ba77-a919-4c1b-aa58-c6649dc6928d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:42.464453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 648/1293] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:42 2022] POST /identity/v3/projects => generated 357 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:42.464453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bcfde5-1b77-4932-8299-141c5875218d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:42.569455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11ba77-a919-4c1b-aa58-c6649dc6928d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:42.606440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bcfde5-1b77-4932-8299-141c5875218d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:42.626198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2fc5f-3406-48c4-ae5a-8534db0e233b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:42.629668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2fc5f-3406-48c4-ae5a-8534db0e233b admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:42.629959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2fc5f-3406-48c4-ae5a-8534db0e233b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:42.630263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2fc5f-3406-48c4-ae5a-8534db0e233b admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:42.634224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2fc5f-3406-48c4-ae5a-8534db0e233b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:42.636799 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2fc5f-3406-48c4-ae5a-8534db0e233b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:42.685396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 649/1294] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:42 2022] POST /identity/v3/users => generated 309 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:42.685396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9034f-d1c1-49b4-ae59-046af66aed3c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:42.685396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9034f-d1c1-49b4-ae59-046af66aed3c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:42.685396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9034f-d1c1-49b4-ae59-046af66aed3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:42.701343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ee4b0-10ce-4944-b4c9-6ca96bb46c9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:42.702077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ee4b0-10ce-4944-b4c9-6ca96bb46c9e service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:42.702963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ee4b0-10ce-4944-b4c9-6ca96bb46c9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:42.702963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ee4b0-10ce-4944-b4c9-6ca96bb46c9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:42.733305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:42.746621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4bc9034f-d1c1-49b4-ae59-046af66aed3c None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:42.757522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4bc9034f-d1c1-49b4-ae59-046af66aed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:42.846014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ee4b0-10ce-4944-b4c9-6ca96bb46c9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:42.848650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ee4b0-10ce-4944-b4c9-6ca96bb46c9e service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:42.866243 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 646/1295] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:42.867442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ee4b0-10ce-4944-b4c9-6ca96bb46c9e service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:42.906491 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 650/1296] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Oct 24 13:41:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:42.907440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9034f-d1c1-49b4-ae59-046af66aed3c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:42.952779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ee4b0-10ce-4944-b4c9-6ca96bb46c9e service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:42.958684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c04e1-3057-4645-9de5-10a3151813c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:42.959548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c04e1-3057-4645-9de5-10a3151813c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:42.960330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c04e1-3057-4645-9de5-10a3151813c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:42.960685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c04e1-3057-4645-9de5-10a3151813c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:42.973841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c04e1-3057-4645-9de5-10a3151813c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:42.977398 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c04e1-3057-4645-9de5-10a3151813c1 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:42.992074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 647/1297] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:42.994070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c04e1-3057-4645-9de5-10a3151813c1 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:43.003550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c04e1-3057-4645-9de5-10a3151813c1 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:43.004087 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:43.010314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d9240-fbbb-412a-8900-5b9b9e428323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:43.011481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d9240-fbbb-412a-8900-5b9b9e428323 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.011864 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d9240-fbbb-412a-8900-5b9b9e428323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.012305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d9240-fbbb-412a-8900-5b9b9e428323 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.013056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d9240-fbbb-412a-8900-5b9b9e428323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:43.015109 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d9240-fbbb-412a-8900-5b9b9e428323 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:43.037095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9034f-d1c1-49b4-ae59-046af66aed3c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:43.041270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d191fe0-362f-4808-a51c-f56785e6fbd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:43.042291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d191fe0-362f-4808-a51c-f56785e6fbd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.042576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d191fe0-362f-4808-a51c-f56785e6fbd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.042845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d191fe0-362f-4808-a51c-f56785e6fbd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.048498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 648/1298] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:42 2022] POST /identity/v3/users => generated 285 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:43.050581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d9240-fbbb-412a-8900-5b9b9e428323 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:43.057789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d191fe0-362f-4808-a51c-f56785e6fbd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:43.059503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d191fe0-362f-4808-a51c-f56785e6fbd7 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:43.075414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 651/1299] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:43.076483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d191fe0-362f-4808-a51c-f56785e6fbd7 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:43.078236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d9240-fbbb-412a-8900-5b9b9e428323 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:43.092684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d191fe0-362f-4808-a51c-f56785e6fbd7 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:43.093300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b3731-c80e-41c5-b584-92b05747075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:43.094030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b3731-c80e-41c5-b584-92b05747075f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.094455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b3731-c80e-41c5-b584-92b05747075f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.094825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b3731-c80e-41c5-b584-92b05747075f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.096808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b3731-c80e-41c5-b584-92b05747075f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:43.099188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b3731-c80e-41c5-b584-92b05747075f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:43.104141 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774dd2d-4a9e-4453-acbb-53e440068766 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:43.105164 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774dd2d-4a9e-4453-acbb-53e440068766 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.105682 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774dd2d-4a9e-4453-acbb-53e440068766 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.106039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774dd2d-4a9e-4453-acbb-53e440068766 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.130258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 649/1300] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:43.130258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b3731-c80e-41c5-b584-92b05747075f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:43.156585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b3731-c80e-41c5-b584-92b05747075f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:43.177297 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376665a2-0c63-4645-b28c-599633f4204a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:43.178183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376665a2-0c63-4645-b28c-599633f4204a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.178531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376665a2-0c63-4645-b28c-599633f4204a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.179019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376665a2-0c63-4645-b28c-599633f4204a admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.179788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376665a2-0c63-4645-b28c-599633f4204a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:43.185380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376665a2-0c63-4645-b28c-599633f4204a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:43.209567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 650/1301] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:43.210616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376665a2-0c63-4645-b28c-599633f4204a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:43.232152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376665a2-0c63-4645-b28c-599633f4204a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:43.240913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774dd2d-4a9e-4453-acbb-53e440068766 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:43.243061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774dd2d-4a9e-4453-acbb-53e440068766 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:43.253922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34242612-5494-4f32-bbef-03a8ea95f7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:43.254924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34242612-5494-4f32-bbef-03a8ea95f7d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.255244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 652/1302] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:43.255381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34242612-5494-4f32-bbef-03a8ea95f7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.255747 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34242612-5494-4f32-bbef-03a8ea95f7d7 admin admin] PATH_INFO: `/v3/projects/fcbeaf631cfd4f919ce23471dcbd335e/users/aa547469be1c4485824f1a11fc828615/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.270410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774dd2d-4a9e-4453-acbb-53e440068766 service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:43.302783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34242612-5494-4f32-bbef-03a8ea95f7d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbeaf631cfd4f919ce23471dcbd335e, user_id=aa547469be1c4485824f1a11fc828615, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:43.309953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34242612-5494-4f32-bbef-03a8ea95f7d7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:43.332897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774dd2d-4a9e-4453-acbb-53e440068766 service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:43.343839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 651/1303] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:43 2022] PUT /identity/v3/projects/fcbeaf631cfd4f919ce23471dcbd335e/users/aa547469be1c4485824f1a11fc828615/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:43.349032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34242612-5494-4f32-bbef-03a8ea95f7d7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:43.501282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:43.512938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a34d35-d7b2-4b2b-9dca-7365cfe6b212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:43.517102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a34d35-d7b2-4b2b-9dca-7365cfe6b212 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.518387 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a34d35-d7b2-4b2b-9dca-7365cfe6b212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.519298 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a34d35-d7b2-4b2b-9dca-7365cfe6b212 admin admin] PATH_INFO: `/v3/projects/d4373f323c6345558ce0e9d1195f2776/users/64f259a6b39f4df7b52f6c59e8657c3a/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.532876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34242612-5494-4f32-bbef-03a8ea95f7d7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:43.545730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bdd0b5-852a-4ad2-8842-ce47a5297e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:43.547153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bdd0b5-852a-4ad2-8842-ce47a5297e33 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.547546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bdd0b5-852a-4ad2-8842-ce47a5297e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.547923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bdd0b5-852a-4ad2-8842-ce47a5297e33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.548686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bdd0b5-852a-4ad2-8842-ce47a5297e33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:43.551184 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bdd0b5-852a-4ad2-8842-ce47a5297e33 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:43.560298 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a34d35-d7b2-4b2b-9dca-7365cfe6b212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4373f323c6345558ce0e9d1195f2776, user_id=64f259a6b39f4df7b52f6c59e8657c3a, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:43.564160 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a34d35-d7b2-4b2b-9dca-7365cfe6b212 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:43.578577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 652/1304] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:43.579738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bdd0b5-852a-4ad2-8842-ce47a5297e33 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:43.597332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 653/1305] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:43 2022] PUT /identity/v3/projects/d4373f323c6345558ce0e9d1195f2776/users/64f259a6b39f4df7b52f6c59e8657c3a/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:43.601243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a34d35-d7b2-4b2b-9dca-7365cfe6b212 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:43.604088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bdd0b5-852a-4ad2-8842-ce47a5297e33 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:43.750156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb7676-a459-466e-9860-b338e4698d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:43.751004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb7676-a459-466e-9860-b338e4698d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.751393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb7676-a459-466e-9860-b338e4698d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.751773 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb7676-a459-466e-9860-b338e4698d21 admin admin] PATH_INFO: `/v3/projects/fcbeaf631cfd4f919ce23471dcbd335e/users/aa547469be1c4485824f1a11fc828615/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.759217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb7676-a459-466e-9860-b338e4698d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbeaf631cfd4f919ce23471dcbd335e, user_id=aa547469be1c4485824f1a11fc828615, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:43.760525 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a34d35-d7b2-4b2b-9dca-7365cfe6b212 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:43.762349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb7676-a459-466e-9860-b338e4698d21 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:43.770518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38911a4-bb02-43cb-8ca6-e39f3477e1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:43.771260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38911a4-bb02-43cb-8ca6-e39f3477e1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.771597 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38911a4-bb02-43cb-8ca6-e39f3477e1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.771878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38911a4-bb02-43cb-8ca6-e39f3477e1f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.772536 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38911a4-bb02-43cb-8ca6-e39f3477e1f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:43.778463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38911a4-bb02-43cb-8ca6-e39f3477e1f0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:43.782737 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:43.789191 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 653/1306] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:43 2022] PUT /identity/v3/projects/fcbeaf631cfd4f919ce23471dcbd335e/users/aa547469be1c4485824f1a11fc828615/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:43.800171 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991339fa-ae49-4a2c-b2f7-0c623294662e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.800887 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991339fa-ae49-4a2c-b2f7-0c623294662e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.800887 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991339fa-ae49-4a2c-b2f7-0c623294662e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.810814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 654/1307] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:41:43.814593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38911a4-bb02-43cb-8ca6-e39f3477e1f0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:43.833256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38911a4-bb02-43cb-8ca6-e39f3477e1f0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:43.883770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-991339fa-ae49-4a2c-b2f7-0c623294662e None None] MFA Rules not processed for user `aa547469be1c4485824f1a11fc828615`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:43.885162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-991339fa-ae49-4a2c-b2f7-0c623294662e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:43.924292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c1acf-c3af-4c82-b34b-57345ccc28e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:43.925217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c1acf-c3af-4c82-b34b-57345ccc28e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.925629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c1acf-c3af-4c82-b34b-57345ccc28e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.925984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c1acf-c3af-4c82-b34b-57345ccc28e1 admin admin] PATH_INFO: `/v3/projects/d4373f323c6345558ce0e9d1195f2776/users/64f259a6b39f4df7b52f6c59e8657c3a/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:43.928110 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c1acf-c3af-4c82-b34b-57345ccc28e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4373f323c6345558ce0e9d1195f2776, user_id=64f259a6b39f4df7b52f6c59e8657c3a, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:43.931277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c1acf-c3af-4c82-b34b-57345ccc28e1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:43.970283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 655/1308] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:43 2022] PUT /identity/v3/projects/d4373f323c6345558ce0e9d1195f2776/users/64f259a6b39f4df7b52f6c59e8657c3a/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:41:43.981581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96feb20-da75-4e94-96d1-f5c20ff8423a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:43.984353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96feb20-da75-4e94-96d1-f5c20ff8423a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:43.985667 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96feb20-da75-4e94-96d1-f5c20ff8423a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:44.063126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 654/1309] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:43 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:44.064463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599609f-a0ee-4170-aecc-db43e5060bef None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:44.064754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599609f-a0ee-4170-aecc-db43e5060bef None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:44.065765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599609f-a0ee-4170-aecc-db43e5060bef None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:44.067397 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 655/1310] 10.222.0.20 () {54 vars in 912 bytes} [Mon Oct 24 13:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:41:44.075296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e884761-f21a-41ba-be65-a5f16ef001ef None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:44.075629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e884761-f21a-41ba-be65-a5f16ef001ef None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:44.075985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e884761-f21a-41ba-be65-a5f16ef001ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:44.123728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c96feb20-da75-4e94-96d1-f5c20ff8423a None None] MFA Rules not processed for user `64f259a6b39f4df7b52f6c59e8657c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:44.125220 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c96feb20-da75-4e94-96d1-f5c20ff8423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:44.139111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4e884761-f21a-41ba-be65-a5f16ef001ef None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:44.143173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4e884761-f21a-41ba-be65-a5f16ef001ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:44.308383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 656/1311] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:43 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:44.309300 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96feb20-da75-4e94-96d1-f5c20ff8423a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:44.313697 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:44.337960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 656/1312] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:41:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:44.338963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e884761-f21a-41ba-be65-a5f16ef001ef None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:44.451017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96feb20-da75-4e94-96d1-f5c20ff8423a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:44.460633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03683973-656a-4ae6-9a98-d35a79391512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:44.461285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03683973-656a-4ae6-9a98-d35a79391512 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:44.461624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03683973-656a-4ae6-9a98-d35a79391512 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:44.461901 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03683973-656a-4ae6-9a98-d35a79391512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:44.466959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e884761-f21a-41ba-be65-a5f16ef001ef None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:44.475963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8279-249e-4413-9920-12305bc1d369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:44.477220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8279-249e-4413-9920-12305bc1d369 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:44.478225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8279-249e-4413-9920-12305bc1d369 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:44.478526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd8279-249e-4413-9920-12305bc1d369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:44.504553 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd8279-249e-4413-9920-12305bc1d369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:44.506215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd8279-249e-4413-9920-12305bc1d369 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:44.534793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 657/1313] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:44 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:44.543280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:44.598476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03683973-656a-4ae6-9a98-d35a79391512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:44.600409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03683973-656a-4ae6-9a98-d35a79391512 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:44.632326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8279-249e-4413-9920-12305bc1d369 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:44.645770 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 657/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:44 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:44.648739 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd8279-249e-4413-9920-12305bc1d369 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:44.654996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace4ca0-ae82-420b-baaf-2f373767b9c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:44.656635 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace4ca0-ae82-420b-baaf-2f373767b9c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:44.657163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace4ca0-ae82-420b-baaf-2f373767b9c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:44.663824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace4ca0-ae82-420b-baaf-2f373767b9c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:44.690694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace4ca0-ae82-420b-baaf-2f373767b9c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:44.693069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace4ca0-ae82-420b-baaf-2f373767b9c2 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:44.726922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 658/1315] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:41:44.889665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03683973-656a-4ae6-9a98-d35a79391512 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:44.917141 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03683973-656a-4ae6-9a98-d35a79391512 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:44.925461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cec132-b9e9-421a-9122-c3c04da05e83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:44.926521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cec132-b9e9-421a-9122-c3c04da05e83 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:44.926947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cec132-b9e9-421a-9122-c3c04da05e83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:44.927388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cec132-b9e9-421a-9122-c3c04da05e83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:45.091057 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cec132-b9e9-421a-9122-c3c04da05e83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:45.095957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cec132-b9e9-421a-9122-c3c04da05e83 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:45.129507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 658/1316] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:45.646698 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace4ca0-ae82-420b-baaf-2f373767b9c2 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:45.867370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace4ca0-ae82-420b-baaf-2f373767b9c2 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:45.880201 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:45.881442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:45.881962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:45.882494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] PATH_INFO: `/v3/users/7c7c1846407a45aa812ff0e59ba8797b` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:45.887101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c7c1846407a45aa812ff0e59ba8797b)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:45.889364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: WARNING py.warnings [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:41:45.889364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: warnings.warn(msg) Oct 24 13:41:45.889364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: Oct 24 13:41:45.899689 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:46.065920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c7c1846407a45aa812ff0e59ba8797b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:46.070594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c7c1846407a45aa812ff0e59ba8797b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:46.075982 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c7c1846407a45aa812ff0e59ba8797b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:46.112310 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c7c1846407a45aa812ff0e59ba8797b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:46.171560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d4dda24c-c577-4b43-9b4d-119001b34982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c7c1846407a45aa812ff0e59ba8797b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:46.185783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65cfc4-fc6c-4d0f-b148-0078cdcc2e74 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:46.186972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65cfc4-fc6c-4d0f-b148-0078cdcc2e74 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:46.187533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65cfc4-fc6c-4d0f-b148-0078cdcc2e74 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:46.194698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 659/1317] 10.222.0.20 () {54 vars in 961 bytes} [Mon Oct 24 13:41:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:41:46.215944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 659/1318] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:41:45 2022] DELETE /identity/v3/users/7c7c1846407a45aa812ff0e59ba8797b => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:46.222267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9f73b-3671-40e5-8bdd-b46ff0030b0f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:46.226419 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62aeb4d-670f-4eb4-8847-cbee90d070d3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:46.226924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62aeb4d-670f-4eb4-8847-cbee90d070d3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:46.227384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62aeb4d-670f-4eb4-8847-cbee90d070d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:46.235015 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9f73b-3671-40e5-8bdd-b46ff0030b0f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:46.235434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9f73b-3671-40e5-8bdd-b46ff0030b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:46.278730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a62aeb4d-670f-4eb4-8847-cbee90d070d3 None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:46.286706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a0c9f73b-3671-40e5-8bdd-b46ff0030b0f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:46.287490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a62aeb4d-670f-4eb4-8847-cbee90d070d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:46.289469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a0c9f73b-3671-40e5-8bdd-b46ff0030b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:46.305160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:46.441458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 660/1319] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Oct 24 13:41:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:46.447740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1b0a7-b0d5-45d9-9083-0dc3d88ed462 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:46.448364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1b0a7-b0d5-45d9-9083-0dc3d88ed462 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:46.448865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1b0a7-b0d5-45d9-9083-0dc3d88ed462 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:46.452690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 661/1320] 10.222.0.20 () {54 vars in 912 bytes} [Mon Oct 24 13:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:41:46.458456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513e94-bc91-4056-966f-3b4a82374a12 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:46.459720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513e94-bc91-4056-966f-3b4a82374a12 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:46.460634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f513e94-bc91-4056-966f-3b4a82374a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:46.477428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 660/1321] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:46.497751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7f513e94-bc91-4056-966f-3b4a82374a12 None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:46.499142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7f513e94-bc91-4056-966f-3b4a82374a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:46.512839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 662/1322] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:41:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:46.520185 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9f73b-3671-40e5-8bdd-b46ff0030b0f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:46.523638 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a0c9f73b-3671-40e5-8bdd-b46ff0030b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:46.536577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f513e94-bc91-4056-966f-3b4a82374a12 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:46.554992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7f513e94-bc91-4056-966f-3b4a82374a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:46.687333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9f73b-3671-40e5-8bdd-b46ff0030b0f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:46.700149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccea720-94ce-4a33-8450-13945f546c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:46.700795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccea720-94ce-4a33-8450-13945f546c56 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:46.701079 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccea720-94ce-4a33-8450-13945f546c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:46.701351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccea720-94ce-4a33-8450-13945f546c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:46.728848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccea720-94ce-4a33-8450-13945f546c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:46.729632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f513e94-bc91-4056-966f-3b4a82374a12 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:46.731023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccea720-94ce-4a33-8450-13945f546c56 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:46.738197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5b78f-caef-4051-91cc-094c6c50a547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:46.739755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5b78f-caef-4051-91cc-094c6c50a547 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:46.740415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5b78f-caef-4051-91cc-094c6c50a547 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:46.740893 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5b78f-caef-4051-91cc-094c6c50a547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:46.747902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 661/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:46.785007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5b78f-caef-4051-91cc-094c6c50a547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:46.786575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5b78f-caef-4051-91cc-094c6c50a547 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:46.827310 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 663/1324] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:48.622635 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccea720-94ce-4a33-8450-13945f546c56 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:48.625454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8ccea720-94ce-4a33-8450-13945f546c56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:48.668607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccea720-94ce-4a33-8450-13945f546c56 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:48.701504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:48.706056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:48.706793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:48.707082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] PATH_INFO: `/v3/projects/fabcd819e2e941118fc937524f00d5ac` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:48.712630 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fabcd819e2e941118fc937524f00d5ac)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:48.713758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: WARNING py.warnings [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:41:48.713758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: warnings.warn(msg) Oct 24 13:41:48.713758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: Oct 24 13:41:48.719342 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:48.723380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:48.921041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fabcd819e2e941118fc937524f00d5ac', 'request_id': 'req-c5773940-7de6-47d3-8cce-94eaf44b57e4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:48.941799 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fabcd819e2e941118fc937524f00d5ac', 'request_id': 'req-c5773940-7de6-47d3-8cce-94eaf44b57e4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:48.962992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fabcd819e2e941118fc937524f00d5ac', 'request_id': 'req-c5773940-7de6-47d3-8cce-94eaf44b57e4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:49.003779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] The token cache is being invalidate because project fabcd819e2e941118fc937524f00d5ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:41:49.004057 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:49.006040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c5773940-7de6-47d3-8cce-94eaf44b57e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:49.011800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 662/1325] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:41:48 2022] DELETE /identity/v3/projects/fabcd819e2e941118fc937524f00d5ac => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:49.015506 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5b78f-caef-4051-91cc-094c6c50a547 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:49.031158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5b78f-caef-4051-91cc-094c6c50a547 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:49.053275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e72bb1-0a1f-4e01-8d21-40b2b69d3ae0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:49.053569 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e72bb1-0a1f-4e01-8d21-40b2b69d3ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:49.053852 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e72bb1-0a1f-4e01-8d21-40b2b69d3ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:49.112643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-89e72bb1-0a1f-4e01-8d21-40b2b69d3ae0 None None] MFA Rules not processed for user `1ea8feefb0614972898c1b1ad28077a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:49.113916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-89e72bb1-0a1f-4e01-8d21-40b2b69d3ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:49.133545 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:49.134605 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:49.135289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:49.135990 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] PATH_INFO: `/v3/users/aa547469be1c4485824f1a11fc828615` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:49.144288 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa547469be1c4485824f1a11fc828615)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:49.151200 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:49.248155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:49.263008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa547469be1c4485824f1a11fc828615'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:49.268006 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa547469be1c4485824f1a11fc828615'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:49.268878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa547469be1c4485824f1a11fc828615'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:49.278333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 663/1326] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:49.291672 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa547469be1c4485824f1a11fc828615'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:49.301400 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e72bb1-0a1f-4e01-8d21-40b2b69d3ae0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:49.314470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-89e72bb1-0a1f-4e01-8d21-40b2b69d3ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:49.325100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa547469be1c4485824f1a11fc828615'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:49.351449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 664/1327] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:41:49 2022] DELETE /identity/v3/users/aa547469be1c4485824f1a11fc828615 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:41:49.451122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e72bb1-0a1f-4e01-8d21-40b2b69d3ae0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:49.474235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:49.554653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ccfff-eaab-4b7e-aa69-82362c2f6e3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:49.555462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ccfff-eaab-4b7e-aa69-82362c2f6e3c service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:49.555841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ccfff-eaab-4b7e-aa69-82362c2f6e3c service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:49.556255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ccfff-eaab-4b7e-aa69-82362c2f6e3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:49.558288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6d8ccfff-eaab-4b7e-aa69-82362c2f6e3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:49.635236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ccfff-eaab-4b7e-aa69-82362c2f6e3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:49.637566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ccfff-eaab-4b7e-aa69-82362c2f6e3c service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:49.651144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 664/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:51.132104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:51.133672 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:51.284247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46ec2e-7ce3-4fb4-a206-47379a6a93a8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:51.303524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:51.304417 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:51.305424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:51.305705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] PATH_INFO: `/v3/projects/fcbeaf631cfd4f919ce23471dcbd335e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:51.310393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcbeaf631cfd4f919ce23471dcbd335e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:51.312924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:51.443231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcbeaf631cfd4f919ce23471dcbd335e', 'request_id': 'req-216a5878-46a2-492c-9ecd-adc6509bcd33'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:51.472762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcbeaf631cfd4f919ce23471dcbd335e', 'request_id': 'req-216a5878-46a2-492c-9ecd-adc6509bcd33'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:51.482278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcbeaf631cfd4f919ce23471dcbd335e', 'request_id': 'req-216a5878-46a2-492c-9ecd-adc6509bcd33'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:51.516421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] The token cache is being invalidate because project fcbeaf631cfd4f919ce23471dcbd335e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:41:51.517055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:51.518007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-216a5878-46a2-492c-9ecd-adc6509bcd33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:51.520390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 665/1329] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:41:51 2022] DELETE /identity/v3/projects/fcbeaf631cfd4f919ce23471dcbd335e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:51.527397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c3f45-5f08-4172-a84f-ef7fae6e9629 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:51.528319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c3f45-5f08-4172-a84f-ef7fae6e9629 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:51.528814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c3f45-5f08-4172-a84f-ef7fae6e9629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:51.573722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fe5c3f45-5f08-4172-a84f-ef7fae6e9629 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:51.575232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fe5c3f45-5f08-4172-a84f-ef7fae6e9629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:51.727095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 665/1330] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:51.736178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39567eff-f7d8-4408-b23d-a0887b93a28a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:51.737399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39567eff-f7d8-4408-b23d-a0887b93a28a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:51.737399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39567eff-f7d8-4408-b23d-a0887b93a28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:51.776083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-39567eff-f7d8-4408-b23d-a0887b93a28a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:51.777812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-39567eff-f7d8-4408-b23d-a0887b93a28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:51.803699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 666/1331] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:51.807951 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c3f45-5f08-4172-a84f-ef7fae6e9629 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:51.823823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c3f45-5f08-4172-a84f-ef7fae6e9629 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:51.839647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8d54a-4f2f-4f79-8e2d-4b44b32cae59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:51.840284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8d54a-4f2f-4f79-8e2d-4b44b32cae59 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:51.840568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8d54a-4f2f-4f79-8e2d-4b44b32cae59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:51.841996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8d54a-4f2f-4f79-8e2d-4b44b32cae59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:51.842936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8d54a-4f2f-4f79-8e2d-4b44b32cae59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:41:51.843246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8d54a-4f2f-4f79-8e2d-4b44b32cae59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:51.848827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8d54a-4f2f-4f79-8e2d-4b44b32cae59 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:51.897618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 666/1332] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:41:51.902354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39567eff-f7d8-4408-b23d-a0887b93a28a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:51.918517 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39567eff-f7d8-4408-b23d-a0887b93a28a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:51.930858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2d7ba-2326-4ce8-9234-c37f191345bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:51.931500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d7ba-2326-4ce8-9234-c37f191345bf admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:51.931766 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d7ba-2326-4ce8-9234-c37f191345bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:51.932111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d7ba-2326-4ce8-9234-c37f191345bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:51.934974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2d7ba-2326-4ce8-9234-c37f191345bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:51.945458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2d7ba-2326-4ce8-9234-c37f191345bf admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:52.006443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 667/1333] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:51 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:52.011837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8d54a-4f2f-4f79-8e2d-4b44b32cae59 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:52.026972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8d54a-4f2f-4f79-8e2d-4b44b32cae59 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:52.152953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:52.186532 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769bd06-f9a3-4e2f-b27e-b2ab6e29feb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:52.187158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769bd06-f9a3-4e2f-b27e-b2ab6e29feb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:52.187430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769bd06-f9a3-4e2f-b27e-b2ab6e29feb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:52.187700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769bd06-f9a3-4e2f-b27e-b2ab6e29feb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:52.188346 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769bd06-f9a3-4e2f-b27e-b2ab6e29feb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:52.196406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769bd06-f9a3-4e2f-b27e-b2ab6e29feb7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:52.257619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 667/1334] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:52 2022] POST /identity/v3/users => generated 291 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:52.262158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2d7ba-2326-4ce8-9234-c37f191345bf admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:52.284757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2d7ba-2326-4ce8-9234-c37f191345bf admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:52.302755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b32f94-4967-4db3-ac1d-8a467180c060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:52.305326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b32f94-4967-4db3-ac1d-8a467180c060 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:52.305607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b32f94-4967-4db3-ac1d-8a467180c060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:52.305877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b32f94-4967-4db3-ac1d-8a467180c060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:52.307583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b32f94-4967-4db3-ac1d-8a467180c060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:52.314297 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b32f94-4967-4db3-ac1d-8a467180c060 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:52.353062 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 668/1335] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:52.355716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769bd06-f9a3-4e2f-b27e-b2ab6e29feb7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:52.373505 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769bd06-f9a3-4e2f-b27e-b2ab6e29feb7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:52.386747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be93757-c8e1-4be8-98a5-05963c1fdb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:52.389366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93757-c8e1-4be8-98a5-05963c1fdb9c admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:52.389809 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93757-c8e1-4be8-98a5-05963c1fdb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:52.390507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93757-c8e1-4be8-98a5-05963c1fdb9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:52.395381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be93757-c8e1-4be8-98a5-05963c1fdb9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:52.399239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be93757-c8e1-4be8-98a5-05963c1fdb9c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:52.428094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 668/1336] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:52.432515 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b32f94-4967-4db3-ac1d-8a467180c060 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:52.446900 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b32f94-4967-4db3-ac1d-8a467180c060 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:52.467643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76fca5-1f88-45a5-8ce6-b72c10c160c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:52.468338 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76fca5-1f88-45a5-8ce6-b72c10c160c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:52.468610 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76fca5-1f88-45a5-8ce6-b72c10c160c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:52.468882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76fca5-1f88-45a5-8ce6-b72c10c160c1 admin admin] PATH_INFO: `/v3/projects/b1c671a946734eca995cd32ffe52d20b/users/5643765614ab40e2a607fcf22beb60a9/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:52.478640 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:52.515215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76fca5-1f88-45a5-8ce6-b72c10c160c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c671a946734eca995cd32ffe52d20b, user_id=5643765614ab40e2a607fcf22beb60a9, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:52.518986 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76fca5-1f88-45a5-8ce6-b72c10c160c1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:52.552051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 669/1337] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:52 2022] PUT /identity/v3/projects/b1c671a946734eca995cd32ffe52d20b/users/5643765614ab40e2a607fcf22beb60a9/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:52.562313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8b640-8028-4bc4-a195-06bbb3390775 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:52.564166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8b640-8028-4bc4-a195-06bbb3390775 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:52.565596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8b640-8028-4bc4-a195-06bbb3390775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:52.633054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-05f8b640-8028-4bc4-a195-06bbb3390775 None None] MFA Rules not processed for user `5643765614ab40e2a607fcf22beb60a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:52.635065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-05f8b640-8028-4bc4-a195-06bbb3390775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:52.785115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 669/1338] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:52.789393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76fca5-1f88-45a5-8ce6-b72c10c160c1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:52.918529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76fca5-1f88-45a5-8ce6-b72c10c160c1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:52.935140 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974098dd-d2f5-42be-83f9-cd6c48055ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:52.936302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974098dd-d2f5-42be-83f9-cd6c48055ded admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:52.936607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974098dd-d2f5-42be-83f9-cd6c48055ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:52.936880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974098dd-d2f5-42be-83f9-cd6c48055ded admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:52.937544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974098dd-d2f5-42be-83f9-cd6c48055ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:52.948405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974098dd-d2f5-42be-83f9-cd6c48055ded admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:53.005099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 670/1339] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:41:52 2022] POST /identity/v3/projects => generated 379 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:41:53.009270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8b640-8028-4bc4-a195-06bbb3390775 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:53.027353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8b640-8028-4bc4-a195-06bbb3390775 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:53.107837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c651e-a26b-4b8b-b513-288556ec0e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:53.108641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c651e-a26b-4b8b-b513-288556ec0e2b admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:53.108920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c651e-a26b-4b8b-b513-288556ec0e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:53.109195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c651e-a26b-4b8b-b513-288556ec0e2b admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:53.113016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c651e-a26b-4b8b-b513-288556ec0e2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:53.117870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c651e-a26b-4b8b-b513-288556ec0e2b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:53.161286 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 670/1340] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:41:53 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:41:53.162973 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974098dd-d2f5-42be-83f9-cd6c48055ded admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:53.180705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974098dd-d2f5-42be-83f9-cd6c48055ded admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:53.188125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3d904-6d1e-4980-86b6-16ffa00a1be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:53.189283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3d904-6d1e-4980-86b6-16ffa00a1be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:53.189935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3d904-6d1e-4980-86b6-16ffa00a1be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:53.190150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3d904-6d1e-4980-86b6-16ffa00a1be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:53.191144 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3d904-6d1e-4980-86b6-16ffa00a1be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:53.193124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3d904-6d1e-4980-86b6-16ffa00a1be5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:53.219349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 671/1341] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:53.225290 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c651e-a26b-4b8b-b513-288556ec0e2b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:53.260942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c651e-a26b-4b8b-b513-288556ec0e2b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:53.266248 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b9bf6-ed5e-4d37-843a-b97e3d17cb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:53.267153 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b9bf6-ed5e-4d37-843a-b97e3d17cb0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:53.268249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b9bf6-ed5e-4d37-843a-b97e3d17cb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:53.268565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b9bf6-ed5e-4d37-843a-b97e3d17cb0d admin admin] PATH_INFO: `/v3/projects/b9da76fd352a402c8806f94c4e1a75cc/users/0071d5dda27f44299f86256e9e918687/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:53.303337 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b9bf6-ed5e-4d37-843a-b97e3d17cb0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9da76fd352a402c8806f94c4e1a75cc, user_id=0071d5dda27f44299f86256e9e918687, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:53.307618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b9bf6-ed5e-4d37-843a-b97e3d17cb0d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:53.340154 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3d904-6d1e-4980-86b6-16ffa00a1be5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:53.341625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 671/1342] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:53 2022] PUT /identity/v3/projects/b9da76fd352a402c8806f94c4e1a75cc/users/0071d5dda27f44299f86256e9e918687/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:53.353824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3d904-6d1e-4980-86b6-16ffa00a1be5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:53.521209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de091ee1-cafb-44e0-bbfb-a13748325f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:53.521838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de091ee1-cafb-44e0-bbfb-a13748325f69 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:53.522117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de091ee1-cafb-44e0-bbfb-a13748325f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:53.522416 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de091ee1-cafb-44e0-bbfb-a13748325f69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:53.523064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de091ee1-cafb-44e0-bbfb-a13748325f69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:53.524684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de091ee1-cafb-44e0-bbfb-a13748325f69 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:53.582241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b9bf6-ed5e-4d37-843a-b97e3d17cb0d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:53.597793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 672/1343] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:41:53.605170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b9bf6-ed5e-4d37-843a-b97e3d17cb0d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:53.621126 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd32d4c-8018-4241-a6fc-f5d0e1441ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:53.621862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd32d4c-8018-4241-a6fc-f5d0e1441ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:53.622278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd32d4c-8018-4241-a6fc-f5d0e1441ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:53.622556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd32d4c-8018-4241-a6fc-f5d0e1441ba8 admin admin] PATH_INFO: `/v3/projects/b9da76fd352a402c8806f94c4e1a75cc/users/0071d5dda27f44299f86256e9e918687/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:53.635222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd32d4c-8018-4241-a6fc-f5d0e1441ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9da76fd352a402c8806f94c4e1a75cc, user_id=0071d5dda27f44299f86256e9e918687, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:53.644235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd32d4c-8018-4241-a6fc-f5d0e1441ba8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:53.680067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 672/1344] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:41:53 2022] PUT /identity/v3/projects/b9da76fd352a402c8806f94c4e1a75cc/users/0071d5dda27f44299f86256e9e918687/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:53.680303 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365b32a-6679-41e7-a186-9367550353e6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:53.680408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365b32a-6679-41e7-a186-9367550353e6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:53.680623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365b32a-6679-41e7-a186-9367550353e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:53.760228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8365b32a-6679-41e7-a186-9367550353e6 None None] MFA Rules not processed for user `0071d5dda27f44299f86256e9e918687`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:53.765049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8365b32a-6679-41e7-a186-9367550353e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:53.928211 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:54.004740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 673/1345] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:54.016146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd32d4c-8018-4241-a6fc-f5d0e1441ba8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:54.018125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ccd32d4c-8018-4241-a6fc-f5d0e1441ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:54.024352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:54.168903 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd32d4c-8018-4241-a6fc-f5d0e1441ba8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:54.185321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1f2e6-197d-488f-9342-eab906b66adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:54.186186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1f2e6-197d-488f-9342-eab906b66adb service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:54.186307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1f2e6-197d-488f-9342-eab906b66adb service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:54.186550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1f2e6-197d-488f-9342-eab906b66adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:54.235187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1f2e6-197d-488f-9342-eab906b66adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:54.237187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1f2e6-197d-488f-9342-eab906b66adb service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:54.302101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 673/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:54.421791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365b32a-6679-41e7-a186-9367550353e6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:54.436729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365b32a-6679-41e7-a186-9367550353e6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:54.443535 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa91bb-7142-402a-89c4-f12c65139da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:54.444493 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa91bb-7142-402a-89c4-f12c65139da5 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:54.445104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa91bb-7142-402a-89c4-f12c65139da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:54.445601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa91bb-7142-402a-89c4-f12c65139da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:54.572058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa91bb-7142-402a-89c4-f12c65139da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:54.573660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa91bb-7142-402a-89c4-f12c65139da5 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:54.585068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 674/1347] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:55.681660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d465a-ecaf-45a2-b747-fe06ccd247a4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:55.682207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d465a-ecaf-45a2-b747-fe06ccd247a4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:55.682304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d465a-ecaf-45a2-b747-fe06ccd247a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:55.763217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e47d465a-ecaf-45a2-b747-fe06ccd247a4 None None] MFA Rules not processed for user `1461a94dd8ef48f79a0d8642a1bc1e58`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:55.764759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e47d465a-ecaf-45a2-b747-fe06ccd247a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:55.952827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 674/1348] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:55.966712 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa91bb-7142-402a-89c4-f12c65139da5 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:55.971617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-edaa91bb-7142-402a-89c4-f12c65139da5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:56.083671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeef44-caf7-4960-aacd-c1ab814b0d67 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:56.084395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeef44-caf7-4960-aacd-c1ab814b0d67 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:56.084766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feeef44-caf7-4960-aacd-c1ab814b0d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:56.130970 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-8feeef44-caf7-4960-aacd-c1ab814b0d67 None None] MFA Rules not processed for user `1ea8feefb0614972898c1b1ad28077a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:56.132361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8feeef44-caf7-4960-aacd-c1ab814b0d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:56.285392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa91bb-7142-402a-89c4-f12c65139da5 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:56.313156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafa51b-d801-4ddd-8ccc-339397ad0d86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:56.314802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafa51b-d801-4ddd-8ccc-339397ad0d86 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:56.318834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafa51b-d801-4ddd-8ccc-339397ad0d86 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:56.319253 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafa51b-d801-4ddd-8ccc-339397ad0d86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:56.363714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 675/1349] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:41:56.409892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafa51b-d801-4ddd-8ccc-339397ad0d86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:56.415812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafa51b-d801-4ddd-8ccc-339397ad0d86 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:56.487278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 675/1350] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:41:57.452471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c828ad8-f963-4ada-b423-2bf8c302a0f8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:57.454608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c828ad8-f963-4ada-b423-2bf8c302a0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:57.455039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c828ad8-f963-4ada-b423-2bf8c302a0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:57.513641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5c828ad8-f963-4ada-b423-2bf8c302a0f8 None None] MFA Rules not processed for user `be793f4685f94beba74adedbc02ee50a`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:57.515347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5c828ad8-f963-4ada-b423-2bf8c302a0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:57.592354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:57.700606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 676/1351] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:57.711160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387fe15-8576-4ea6-acd5-9e0acfd93d68 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:57.712065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387fe15-8576-4ea6-acd5-9e0acfd93d68 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:57.712508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387fe15-8576-4ea6-acd5-9e0acfd93d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:57.747409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e387fe15-8576-4ea6-acd5-9e0acfd93d68 None None] MFA Rules not processed for user `be793f4685f94beba74adedbc02ee50a`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:57.748744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e387fe15-8576-4ea6-acd5-9e0acfd93d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:57.762479 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 676/1352] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:57.876724 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c828ad8-f963-4ada-b423-2bf8c302a0f8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:57.878622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5c828ad8-f963-4ada-b423-2bf8c302a0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:57.881829 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387fe15-8576-4ea6-acd5-9e0acfd93d68 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:57.908289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387fe15-8576-4ea6-acd5-9e0acfd93d68 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:57.926440 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a71ca1-1154-4711-b2f0-a9ec2963d89e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:57.928193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a71ca1-1154-4711-b2f0-a9ec2963d89e service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:57.929026 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a71ca1-1154-4711-b2f0-a9ec2963d89e service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:57.929699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a71ca1-1154-4711-b2f0-a9ec2963d89e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:57.947317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a71ca1-1154-4711-b2f0-a9ec2963d89e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:57.949769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a71ca1-1154-4711-b2f0-a9ec2963d89e service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:57.970591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 677/1353] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:58.073053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c828ad8-f963-4ada-b423-2bf8c302a0f8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:58.079774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:58.080798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:58.081393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:58.081887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] PATH_INFO: `/v3/users/59988220318042cab2f81f9ed3a57caa` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:58.083330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59988220318042cab2f81f9ed3a57caa)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:58.085467 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:58.161939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59988220318042cab2f81f9ed3a57caa'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:58.162899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59988220318042cab2f81f9ed3a57caa'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:58.185732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59988220318042cab2f81f9ed3a57caa'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:58.187333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59988220318042cab2f81f9ed3a57caa'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:58.215807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59988220318042cab2f81f9ed3a57caa'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:41:58.248165 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 677/1354] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:41:57 2022] DELETE /identity/v3/users/59988220318042cab2f81f9ed3a57caa => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:41:58.251387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f38bd6-fd1d-448a-b6f3-207b28fd7e8c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:58.251800 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f38bd6-fd1d-448a-b6f3-207b28fd7e8c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:58.252757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f38bd6-fd1d-448a-b6f3-207b28fd7e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:58.294128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-80f38bd6-fd1d-448a-b6f3-207b28fd7e8c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:58.295499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-80f38bd6-fd1d-448a-b6f3-207b28fd7e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:58.385432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:58.447128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 678/1355] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:58.479135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:58.482226 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:58.517053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48a924-3014-476f-a9e1-aec7efcf17d1 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:58.519366 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48a924-3014-476f-a9e1-aec7efcf17d1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:58.519841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48a924-3014-476f-a9e1-aec7efcf17d1 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:58.523316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 679/1356] 10.222.0.20 () {54 vars in 912 bytes} [Mon Oct 24 13:41:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:41:58.534414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80b520-e82b-4b47-92cf-a405cfec4b7e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:58.536906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80b520-e82b-4b47-92cf-a405cfec4b7e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:58.542573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80b520-e82b-4b47-92cf-a405cfec4b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:58.651340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4e80b520-e82b-4b47-92cf-a405cfec4b7e None None] MFA Rules not processed for user `5b8479c571ec449fbfc306d25fbf7fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:58.652656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4e80b520-e82b-4b47-92cf-a405cfec4b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:58.660447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:58.668232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 680/1357] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:58.669839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229bde1-ca92-48b6-9ea8-8e65f8899f7c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:58.670136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229bde1-ca92-48b6-9ea8-8e65f8899f7c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:58.670504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229bde1-ca92-48b6-9ea8-8e65f8899f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:58.700615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea5bed-3018-4993-aaa2-6a687f0b6852 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:58.712138 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d7a41-8f3f-4a3c-a009-e57e1174723e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:58.714321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d7a41-8f3f-4a3c-a009-e57e1174723e service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:58.714618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d7a41-8f3f-4a3c-a009-e57e1174723e service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:58.714897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d7a41-8f3f-4a3c-a009-e57e1174723e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:58.740283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c229bde1-ca92-48b6-9ea8-8e65f8899f7c None None] MFA Rules not processed for user `9cfcee4879df4843867f69c5091df370`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:58.741599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c229bde1-ca92-48b6-9ea8-8e65f8899f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:58.758303 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d7a41-8f3f-4a3c-a009-e57e1174723e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:58.766822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d7a41-8f3f-4a3c-a009-e57e1174723e service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:58.801976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 678/1358] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:58.803717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05047910-d81a-4be6-ab5f-0a42ce7e24e3 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:58.804191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05047910-d81a-4be6-ab5f-0a42ce7e24e3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:58.804790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05047910-d81a-4be6-ab5f-0a42ce7e24e3 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:58.809659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 679/1359] 10.222.0.20 () {54 vars in 912 bytes} [Mon Oct 24 13:41:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:41:58.811439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05047910-d81a-4be6-ab5f-0a42ce7e24e3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:58.816048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-05047910-d81a-4be6-ab5f-0a42ce7e24e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:58.849435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05047910-d81a-4be6-ab5f-0a42ce7e24e3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:58.859616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55d7ae-9be3-42a7-b33c-01f7e8bbd42e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:58.860278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55d7ae-9be3-42a7-b33c-01f7e8bbd42e service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:58.860553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55d7ae-9be3-42a7-b33c-01f7e8bbd42e service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:58.860800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55d7ae-9be3-42a7-b33c-01f7e8bbd42e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:58.892216 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55d7ae-9be3-42a7-b33c-01f7e8bbd42e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:58.894943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55d7ae-9be3-42a7-b33c-01f7e8bbd42e service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:58.922542 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 680/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:58.927887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bf406-3986-471a-8797-22ead4cd8e37 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:58.928526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bf406-3986-471a-8797-22ead4cd8e37 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:58.928794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bf406-3986-471a-8797-22ead4cd8e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:58.957377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 681/1361] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:58.970006 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2816238-49da-4ea9-ad81-19c6039d78ba None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:58.971277 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2816238-49da-4ea9-ad81-19c6039d78ba None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:58.971911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2816238-49da-4ea9-ad81-19c6039d78ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:58.996452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c70bf406-3986-471a-8797-22ead4cd8e37 None None] MFA Rules not processed for user `5b8479c571ec449fbfc306d25fbf7fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:58.997803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c70bf406-3986-471a-8797-22ead4cd8e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:59.011636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a2816238-49da-4ea9-ad81-19c6039d78ba None None] MFA Rules not processed for user `9cfcee4879df4843867f69c5091df370`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:59.012860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a2816238-49da-4ea9-ad81-19c6039d78ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:59.015253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 681/1362] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:59.026850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 682/1363] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:59.067810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bf406-3986-471a-8797-22ead4cd8e37 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:59.107042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bf406-3986-471a-8797-22ead4cd8e37 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:59.118543 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7da3c-c44d-465a-8c30-4e83b4c0a4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:59.119705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7da3c-c44d-465a-8c30-4e83b4c0a4cb service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:59.119855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7da3c-c44d-465a-8c30-4e83b4c0a4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:59.120206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7da3c-c44d-465a-8c30-4e83b4c0a4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:59.134969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7da3c-c44d-465a-8c30-4e83b4c0a4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:59.136576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7da3c-c44d-465a-8c30-4e83b4c0a4cb service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:59.153910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 682/1364] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:59.640444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2816238-49da-4ea9-ad81-19c6039d78ba None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:59.642071 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a2816238-49da-4ea9-ad81-19c6039d78ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:59.766243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7da3c-c44d-465a-8c30-4e83b4c0a4cb service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:59.768485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e9d7da3c-c44d-465a-8c30-4e83b4c0a4cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:59.800877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2816238-49da-4ea9-ad81-19c6039d78ba None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:59.811379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb163249-0e67-4a6d-b80c-6c70844dc2e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:59.812820 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb163249-0e67-4a6d-b80c-6c70844dc2e9 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:59.813283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb163249-0e67-4a6d-b80c-6c70844dc2e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:59.815400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb163249-0e67-4a6d-b80c-6c70844dc2e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:59.831322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb163249-0e67-4a6d-b80c-6c70844dc2e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:59.833301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb163249-0e67-4a6d-b80c-6c70844dc2e9 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:59.847648 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 683/1365] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:59.958656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb163249-0e67-4a6d-b80c-6c70844dc2e9 service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:59.962332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fb163249-0e67-4a6d-b80c-6c70844dc2e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:00.000140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7da3c-c44d-465a-8c30-4e83b4c0a4cb service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:00.014624 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb163249-0e67-4a6d-b80c-6c70844dc2e9 service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:00.016777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4dde8-d20b-4730-9ecf-82cc449864cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:00.018075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4dde8-d20b-4730-9ecf-82cc449864cf service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:00.018579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4dde8-d20b-4730-9ecf-82cc449864cf service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:00.018945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4dde8-d20b-4730-9ecf-82cc449864cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:00.024778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-01e4dde8-d20b-4730-9ecf-82cc449864cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:00.032433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:00.036823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:00.037296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:00.037720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] PATH_INFO: `/v3/projects/fb21fa4a8de149b29f5324dc2d80464a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:00.040209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb21fa4a8de149b29f5324dc2d80464a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:00.042400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:00.053823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4dde8-d20b-4730-9ecf-82cc449864cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:00.055501 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4dde8-d20b-4730-9ecf-82cc449864cf service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:00.082246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 683/1366] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:42:00.094693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:00.140636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb21fa4a8de149b29f5324dc2d80464a', 'request_id': 'req-57224be6-1eec-4431-a223-07b70fd7aa4b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:00.166517 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb21fa4a8de149b29f5324dc2d80464a', 'request_id': 'req-57224be6-1eec-4431-a223-07b70fd7aa4b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:00.188082 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb21fa4a8de149b29f5324dc2d80464a', 'request_id': 'req-57224be6-1eec-4431-a223-07b70fd7aa4b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:00.223187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] The token cache is being invalidate because project fb21fa4a8de149b29f5324dc2d80464a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:00.224889 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:00.226442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57224be6-1eec-4431-a223-07b70fd7aa4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:00.236103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 684/1367] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:41:59 2022] DELETE /identity/v3/projects/fb21fa4a8de149b29f5324dc2d80464a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:00.238493 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88f94d-29d3-423d-8bd1-ab5355a2e28c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:00.238888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88f94d-29d3-423d-8bd1-ab5355a2e28c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:00.239279 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88f94d-29d3-423d-8bd1-ab5355a2e28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:00.290507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ba88f94d-29d3-423d-8bd1-ab5355a2e28c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:00.293832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ba88f94d-29d3-423d-8bd1-ab5355a2e28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:00.614112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 684/1368] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:00.620969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ab660-eb9e-4abb-9907-7fbef7cb35f7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:00.621309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ab660-eb9e-4abb-9907-7fbef7cb35f7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:00.621592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ab660-eb9e-4abb-9907-7fbef7cb35f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:00.691899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ab2ab660-eb9e-4abb-9907-7fbef7cb35f7 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:00.693159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ab2ab660-eb9e-4abb-9907-7fbef7cb35f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:00.731746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88f94d-29d3-423d-8bd1-ab5355a2e28c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:00.734243 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 685/1369] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:00.749391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88f94d-29d3-423d-8bd1-ab5355a2e28c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:00.780368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d7575-9848-404f-bc71-0f390df22f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:00.781065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d7575-9848-404f-bc71-0f390df22f7a admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:00.781353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d7575-9848-404f-bc71-0f390df22f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:00.781626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d7575-9848-404f-bc71-0f390df22f7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:00.782314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d7575-9848-404f-bc71-0f390df22f7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:00.782616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d7575-9848-404f-bc71-0f390df22f7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:00.785035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d7575-9848-404f-bc71-0f390df22f7a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:00.836900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 685/1370] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:42:00.837567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ab660-eb9e-4abb-9907-7fbef7cb35f7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:00.849350 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ab660-eb9e-4abb-9907-7fbef7cb35f7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:00.873226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a14e00-a690-4a27-a9c6-f45251514c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:00.874783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a14e00-a690-4a27-a9c6-f45251514c43 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:00.875533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a14e00-a690-4a27-a9c6-f45251514c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:00.876014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a14e00-a690-4a27-a9c6-f45251514c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:00.878659 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a14e00-a690-4a27-a9c6-f45251514c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:00.885527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a14e00-a690-4a27-a9c6-f45251514c43 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:00.996072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 686/1371] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:00 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:00.996504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d7575-9848-404f-bc71-0f390df22f7a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:01.012961 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d7575-9848-404f-bc71-0f390df22f7a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:01.171592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:01.250504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce89e26-1c9a-4d0e-b072-fe8607d685b4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:01.250792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce89e26-1c9a-4d0e-b072-fe8607d685b4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:01.251067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce89e26-1c9a-4d0e-b072-fe8607d685b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:01.301669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c848fc2-c625-4776-b985-3236c596daf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:01.302357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c848fc2-c625-4776-b985-3236c596daf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:01.302638 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c848fc2-c625-4776-b985-3236c596daf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:01.302910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c848fc2-c625-4776-b985-3236c596daf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:01.313490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c848fc2-c625-4776-b985-3236c596daf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:01.335650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0ce89e26-1c9a-4d0e-b072-fe8607d685b4 None None] MFA Rules not processed for user `be793f4685f94beba74adedbc02ee50a`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:01.337090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c848fc2-c625-4776-b985-3236c596daf6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:01.344853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0ce89e26-1c9a-4d0e-b072-fe8607d685b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:01.460413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 686/1372] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:00 2022] POST /identity/v3/users => generated 287 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:01.465725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c848fc2-c625-4776-b985-3236c596daf6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:01.529240 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c848fc2-c625-4776-b985-3236c596daf6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:01.571628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320976a-5aae-4bf0-bb02-786ffe8c4e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:01.577962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320976a-5aae-4bf0-bb02-786ffe8c4e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:01.578951 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320976a-5aae-4bf0-bb02-786ffe8c4e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:01.581209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320976a-5aae-4bf0-bb02-786ffe8c4e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:01.584886 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320976a-5aae-4bf0-bb02-786ffe8c4e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:01.597936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320976a-5aae-4bf0-bb02-786ffe8c4e3e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:01.612201 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 687/1373] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:01.660155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce89e26-1c9a-4d0e-b072-fe8607d685b4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:01.674689 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 687/1374] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:01.709069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce89e26-1c9a-4d0e-b072-fe8607d685b4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:01.732591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badf6f4-d0e6-49b5-9017-61b4cba61ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:01.733128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badf6f4-d0e6-49b5-9017-61b4cba61ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:01.733406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badf6f4-d0e6-49b5-9017-61b4cba61ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:01.733678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badf6f4-d0e6-49b5-9017-61b4cba61ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:01.734385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1badf6f4-d0e6-49b5-9017-61b4cba61ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:01.746472 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1badf6f4-d0e6-49b5-9017-61b4cba61ee4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:01.834344 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 688/1375] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:01.837447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320976a-5aae-4bf0-bb02-786ffe8c4e3e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:01.863304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320976a-5aae-4bf0-bb02-786ffe8c4e3e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:01.880440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fc17a-aa90-481d-a588-9fdafbe4ecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:01.884911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fc17a-aa90-481d-a588-9fdafbe4ecaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:01.885490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fc17a-aa90-481d-a588-9fdafbe4ecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:01.887565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fc17a-aa90-481d-a588-9fdafbe4ecaa admin admin] PATH_INFO: `/v3/projects/45bebd4d70fe4613a0c21995128615c3/users/fa9c44dd5b1548d5adc2677b0b46aaab/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:01.960737 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fc17a-aa90-481d-a588-9fdafbe4ecaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=45bebd4d70fe4613a0c21995128615c3, user_id=fa9c44dd5b1548d5adc2677b0b46aaab, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:01.969590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fc17a-aa90-481d-a588-9fdafbe4ecaa admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:02.026738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 688/1376] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:01 2022] PUT /identity/v3/projects/45bebd4d70fe4613a0c21995128615c3/users/fa9c44dd5b1548d5adc2677b0b46aaab/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:02.034541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f999d7-6b5a-4cdb-b60c-62dc8a30d0c2 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:02.035645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f999d7-6b5a-4cdb-b60c-62dc8a30d0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:02.036217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f999d7-6b5a-4cdb-b60c-62dc8a30d0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:02.132849 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b5f999d7-6b5a-4cdb-b60c-62dc8a30d0c2 None None] MFA Rules not processed for user `fa9c44dd5b1548d5adc2677b0b46aaab`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:02.137123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b5f999d7-6b5a-4cdb-b60c-62dc8a30d0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:02.455585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 689/1377] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:02.459830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fc17a-aa90-481d-a588-9fdafbe4ecaa admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:02.678701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fc17a-aa90-481d-a588-9fdafbe4ecaa admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:02.699131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58104bc6-95b1-40de-9f67-e9d00d1ccac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:02.700082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58104bc6-95b1-40de-9f67-e9d00d1ccac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:02.700528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58104bc6-95b1-40de-9f67-e9d00d1ccac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:02.700939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58104bc6-95b1-40de-9f67-e9d00d1ccac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:02.701736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58104bc6-95b1-40de-9f67-e9d00d1ccac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:02.706745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58104bc6-95b1-40de-9f67-e9d00d1ccac1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:02.777663 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 689/1378] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:02 2022] POST /identity/v3/projects => generated 373 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:02.782595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f999d7-6b5a-4cdb-b60c-62dc8a30d0c2 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:02.797663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f999d7-6b5a-4cdb-b60c-62dc8a30d0c2 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:02.811727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:02.951780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a4f37-e817-4bc1-967e-5483220d183a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:02.953637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a4f37-e817-4bc1-967e-5483220d183a admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:02.955688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a4f37-e817-4bc1-967e-5483220d183a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:02.956119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a4f37-e817-4bc1-967e-5483220d183a admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:02.958495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a4f37-e817-4bc1-967e-5483220d183a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:02.964347 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a4f37-e817-4bc1-967e-5483220d183a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:03.028793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58104bc6-95b1-40de-9f67-e9d00d1ccac1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:03.036390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 690/1379] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:02 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:03.045576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58104bc6-95b1-40de-9f67-e9d00d1ccac1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:03.059386 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7339ec4-8b59-4329-9c87-bf6671d7a98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:03.060266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7339ec4-8b59-4329-9c87-bf6671d7a98e admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:03.060730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7339ec4-8b59-4329-9c87-bf6671d7a98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:03.072677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7339ec4-8b59-4329-9c87-bf6671d7a98e admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:03.073850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7339ec4-8b59-4329-9c87-bf6671d7a98e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:03.112880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7339ec4-8b59-4329-9c87-bf6671d7a98e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:03.174956 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a4f37-e817-4bc1-967e-5483220d183a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:03.202931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 690/1380] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:03.210751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a4f37-e817-4bc1-967e-5483220d183a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:03.228494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e5c63-ec45-4bba-b51d-15fd2459da12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:03.229142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e5c63-ec45-4bba-b51d-15fd2459da12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:03.229433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e5c63-ec45-4bba-b51d-15fd2459da12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:03.229723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e5c63-ec45-4bba-b51d-15fd2459da12 admin admin] PATH_INFO: `/v3/projects/5833854955b349f282ddca21961a9068/users/c4c9d4ea00444dc7838d2a9c4577192a/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:03.246899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521de67-8cfd-4de1-9592-c40d2b81d5d2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:03.246899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521de67-8cfd-4de1-9592-c40d2b81d5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:03.247050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521de67-8cfd-4de1-9592-c40d2b81d5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:03.344099 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f521de67-8cfd-4de1-9592-c40d2b81d5d2 None None] MFA Rules not processed for user `1461a94dd8ef48f79a0d8642a1bc1e58`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:03.350607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f521de67-8cfd-4de1-9592-c40d2b81d5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:03.360476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e5c63-ec45-4bba-b51d-15fd2459da12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5833854955b349f282ddca21961a9068, user_id=c4c9d4ea00444dc7838d2a9c4577192a, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:03.374278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e5c63-ec45-4bba-b51d-15fd2459da12 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:03.416552 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 691/1381] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:03 2022] PUT /identity/v3/projects/5833854955b349f282ddca21961a9068/users/c4c9d4ea00444dc7838d2a9c4577192a/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:03.422380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e5c63-ec45-4bba-b51d-15fd2459da12 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:03.591136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 691/1382] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:03.646512 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e5c63-ec45-4bba-b51d-15fd2459da12 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:03.654802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a8345-c357-4367-b2d4-ca5e04a71bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:03.655806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8345-c357-4367-b2d4-ca5e04a71bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:03.656236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8345-c357-4367-b2d4-ca5e04a71bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:03.656647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8345-c357-4367-b2d4-ca5e04a71bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:03.659954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a8345-c357-4367-b2d4-ca5e04a71bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:03.663216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a8345-c357-4367-b2d4-ca5e04a71bc3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:03.689356 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521de67-8cfd-4de1-9592-c40d2b81d5d2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:03.696255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 692/1383] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:03.704647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521de67-8cfd-4de1-9592-c40d2b81d5d2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:03.710043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956506a2-a27b-4925-b188-35dad8f293e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:03.711656 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956506a2-a27b-4925-b188-35dad8f293e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:03.712090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956506a2-a27b-4925-b188-35dad8f293e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:03.712529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956506a2-a27b-4925-b188-35dad8f293e4 admin admin] PATH_INFO: `/v3/projects/5833854955b349f282ddca21961a9068/users/c4c9d4ea00444dc7838d2a9c4577192a/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:03.715351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956506a2-a27b-4925-b188-35dad8f293e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5833854955b349f282ddca21961a9068, user_id=c4c9d4ea00444dc7838d2a9c4577192a, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:03.718634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956506a2-a27b-4925-b188-35dad8f293e4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:03.725694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:03.771331 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 692/1384] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:03 2022] PUT /identity/v3/projects/5833854955b349f282ddca21961a9068/users/c4c9d4ea00444dc7838d2a9c4577192a/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:03.781260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab2577-191a-4430-918d-08cd2639bf08 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:03.781772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab2577-191a-4430-918d-08cd2639bf08 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:03.784438 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab2577-191a-4430-918d-08cd2639bf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:03.847011 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-52ab2577-191a-4430-918d-08cd2639bf08 None None] MFA Rules not processed for user `c4c9d4ea00444dc7838d2a9c4577192a`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:03.849453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-52ab2577-191a-4430-918d-08cd2639bf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:04.023700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:04.049186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 693/1385] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:04.084308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956506a2-a27b-4925-b188-35dad8f293e4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:04.088132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-956506a2-a27b-4925-b188-35dad8f293e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:04.263887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956506a2-a27b-4925-b188-35dad8f293e4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:04.285904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac3dc4-2492-45c9-9f88-f76aede2ee63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:04.287301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac3dc4-2492-45c9-9f88-f76aede2ee63 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:04.287965 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac3dc4-2492-45c9-9f88-f76aede2ee63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:04.288589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac3dc4-2492-45c9-9f88-f76aede2ee63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:04.325241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac3dc4-2492-45c9-9f88-f76aede2ee63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:04.329746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac3dc4-2492-45c9-9f88-f76aede2ee63 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:04.363777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 693/1386] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:04.606698 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab2577-191a-4430-918d-08cd2639bf08 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:04.623036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab2577-191a-4430-918d-08cd2639bf08 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:04.650652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ec410-3dcd-4ccc-a996-c98c6c7f25b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:04.653716 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ec410-3dcd-4ccc-a996-c98c6c7f25b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:04.654326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ec410-3dcd-4ccc-a996-c98c6c7f25b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:04.655002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ec410-3dcd-4ccc-a996-c98c6c7f25b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:04.915420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ec410-3dcd-4ccc-a996-c98c6c7f25b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:04.919200 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ec410-3dcd-4ccc-a996-c98c6c7f25b4 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:04.951954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 694/1387] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:05.454637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c57fd7-5659-47ae-bd46-fb0ef93abc56 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:05.455743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c57fd7-5659-47ae-bd46-fb0ef93abc56 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:05.456234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c57fd7-5659-47ae-bd46-fb0ef93abc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:05.603862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ec410-3dcd-4ccc-a996-c98c6c7f25b4 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:05.605654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-882ec410-3dcd-4ccc-a996-c98c6c7f25b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:05.640420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a9c57fd7-5659-47ae-bd46-fb0ef93abc56 None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:05.646590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a9c57fd7-5659-47ae-bd46-fb0ef93abc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:05.897001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ec410-3dcd-4ccc-a996-c98c6c7f25b4 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:05.899358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 694/1388] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:42:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:05.931539 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:05.938821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:05.939118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:05.952778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] PATH_INFO: `/v3/users/4d877c3fa7524181ae29b4c75b51f330` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:05.962333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d877c3fa7524181ae29b4c75b51f330)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:05.972826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:06.159924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d877c3fa7524181ae29b4c75b51f330'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:06.161365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d877c3fa7524181ae29b4c75b51f330'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:06.162941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d877c3fa7524181ae29b4c75b51f330'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:06.179727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d877c3fa7524181ae29b4c75b51f330'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:06.223813 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d877c3fa7524181ae29b4c75b51f330'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:06.262815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 695/1389] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:05 2022] DELETE /identity/v3/users/4d877c3fa7524181ae29b4c75b51f330 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:06.268529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c71ed-cada-4aa2-aeb8-54dc5ada1455 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:06.268836 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c71ed-cada-4aa2-aeb8-54dc5ada1455 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:06.269094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c71ed-cada-4aa2-aeb8-54dc5ada1455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:06.330547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-436c71ed-cada-4aa2-aeb8-54dc5ada1455 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:06.335925 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-436c71ed-cada-4aa2-aeb8-54dc5ada1455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:06.351859 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:06.354143 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:06.440626 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:06.614822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 695/1390] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:06.690971 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c71ed-cada-4aa2-aeb8-54dc5ada1455 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:06.693275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-436c71ed-cada-4aa2-aeb8-54dc5ada1455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:06.756382 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:06.791745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2607f-524c-4ee9-a00c-9addd3fd30b3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:06.803275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9ded7-b80c-4f4b-95b2-0bb4ce7573cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:06.804645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9ded7-b80c-4f4b-95b2-0bb4ce7573cb service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:06.805104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9ded7-b80c-4f4b-95b2-0bb4ce7573cb service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:06.805527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9ded7-b80c-4f4b-95b2-0bb4ce7573cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:06.814158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-52d9ded7-b80c-4f4b-95b2-0bb4ce7573cb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:06.945602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c71ed-cada-4aa2-aeb8-54dc5ada1455 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:06.973390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fd257-d395-44ca-989c-15c6652c51f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:06.974059 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fd257-d395-44ca-989c-15c6652c51f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:06.974479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fd257-d395-44ca-989c-15c6652c51f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:06.974842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fd257-d395-44ca-989c-15c6652c51f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:07.000911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9ded7-b80c-4f4b-95b2-0bb4ce7573cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:07.001560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9ded7-b80c-4f4b-95b2-0bb4ce7573cb service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:07.026744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fd257-d395-44ca-989c-15c6652c51f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:07.032614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fd257-d395-44ca-989c-15c6652c51f7 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:07.045253 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 696/1391] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:42:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:42:07.061967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 696/1392] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:07.554530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd94a51-eadb-4311-9f7f-869c1856ac41 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:07.554929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd94a51-eadb-4311-9f7f-869c1856ac41 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:07.555101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd94a51-eadb-4311-9f7f-869c1856ac41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:07.662736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6bd94a51-eadb-4311-9f7f-869c1856ac41 None None] MFA Rules not processed for user `5b8479c571ec449fbfc306d25fbf7fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:07.670609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6bd94a51-eadb-4311-9f7f-869c1856ac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:07.707918 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 697/1393] 10.222.0.45 () {58 vars in 1015 bytes} [Mon Oct 24 13:42:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:07.741794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fd257-d395-44ca-989c-15c6652c51f7 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:07.753578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fd257-d395-44ca-989c-15c6652c51f7 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:07.773256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b160353-90cb-4bc7-a177-29e81edadb0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:07.773943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b160353-90cb-4bc7-a177-29e81edadb0f service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:07.774261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b160353-90cb-4bc7-a177-29e81edadb0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:07.774536 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b160353-90cb-4bc7-a177-29e81edadb0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:07.816321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b160353-90cb-4bc7-a177-29e81edadb0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:07.824582 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b160353-90cb-4bc7-a177-29e81edadb0f service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:07.888210 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 697/1394] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:08.015188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd94a51-eadb-4311-9f7f-869c1856ac41 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:08.019858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6bd94a51-eadb-4311-9f7f-869c1856ac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:08.068140 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd94a51-eadb-4311-9f7f-869c1856ac41 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:08.087946 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:08.088774 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:08.090327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:08.090891 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] PATH_INFO: `/v3/users/5643765614ab40e2a607fcf22beb60a9` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:08.093934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5643765614ab40e2a607fcf22beb60a9)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:08.097523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:08.324668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5643765614ab40e2a607fcf22beb60a9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:08.328351 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5643765614ab40e2a607fcf22beb60a9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:08.330343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5643765614ab40e2a607fcf22beb60a9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:08.370505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5643765614ab40e2a607fcf22beb60a9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:08.437122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5643765614ab40e2a607fcf22beb60a9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:08.463348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:08.490075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 698/1395] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:08 2022] DELETE /identity/v3/users/5643765614ab40e2a607fcf22beb60a9 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:08.494844 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf57fb9-db7e-488f-9fba-7ec1de6246cf None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:08.495326 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf57fb9-db7e-488f-9fba-7ec1de6246cf None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:08.495739 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf57fb9-db7e-488f-9fba-7ec1de6246cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:08.570209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3cf57fb9-db7e-488f-9fba-7ec1de6246cf None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:08.575751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3cf57fb9-db7e-488f-9fba-7ec1de6246cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:08.867455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 698/1396] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:09.056187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:09.060551 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:09.302500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67f4d2-0ebd-41cb-ab96-ec0bdbb049c3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:09.313130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9679e-7d5e-44d2-b16f-0f07b26830c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:09.316830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9679e-7d5e-44d2-b16f-0f07b26830c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:09.317402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9679e-7d5e-44d2-b16f-0f07b26830c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:09.318192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9679e-7d5e-44d2-b16f-0f07b26830c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:09.342600 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9679e-7d5e-44d2-b16f-0f07b26830c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:09.344402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9679e-7d5e-44d2-b16f-0f07b26830c4 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:09.377769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 699/1397] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:09.999805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf57fb9-db7e-488f-9fba-7ec1de6246cf None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:10.001323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3cf57fb9-db7e-488f-9fba-7ec1de6246cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:10.061541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf57fb9-db7e-488f-9fba-7ec1de6246cf None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:10.073822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:10.074847 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:10.075397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:10.075824 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] PATH_INFO: `/v3/projects/405f8a6f56404e62bbc875bf8f0b2209` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:10.077880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=405f8a6f56404e62bbc875bf8f0b2209)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:10.080032 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:10.176392 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '405f8a6f56404e62bbc875bf8f0b2209', 'request_id': 'req-7728dec5-cb8f-400d-882e-77f584d50f93'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:10.190813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '405f8a6f56404e62bbc875bf8f0b2209', 'request_id': 'req-7728dec5-cb8f-400d-882e-77f584d50f93'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:10.227093 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '405f8a6f56404e62bbc875bf8f0b2209', 'request_id': 'req-7728dec5-cb8f-400d-882e-77f584d50f93'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:10.252237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] The token cache is being invalidate because project 405f8a6f56404e62bbc875bf8f0b2209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:10.252889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:10.255616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7728dec5-cb8f-400d-882e-77f584d50f93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:10.262979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 699/1398] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:09 2022] DELETE /identity/v3/projects/405f8a6f56404e62bbc875bf8f0b2209 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:10.263174 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465385e5-2286-4a40-b7a2-8a8d52abcb7a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:10.263649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465385e5-2286-4a40-b7a2-8a8d52abcb7a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:10.264218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465385e5-2286-4a40-b7a2-8a8d52abcb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:10.302610 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-465385e5-2286-4a40-b7a2-8a8d52abcb7a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:10.305764 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-465385e5-2286-4a40-b7a2-8a8d52abcb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:10.419206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 700/1399] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:10.428127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b44df6-3373-4b5d-be3d-a9d2affe4d7a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:10.428417 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b44df6-3373-4b5d-be3d-a9d2affe4d7a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:10.428712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b44df6-3373-4b5d-be3d-a9d2affe4d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:10.457427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-43b44df6-3373-4b5d-be3d-a9d2affe4d7a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:10.458821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-43b44df6-3373-4b5d-be3d-a9d2affe4d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:10.471972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 700/1400] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:10.475371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465385e5-2286-4a40-b7a2-8a8d52abcb7a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:10.487463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465385e5-2286-4a40-b7a2-8a8d52abcb7a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:10.493905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1617c0-6edd-45b1-b61e-1f7bff11b1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:10.495005 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1617c0-6edd-45b1-b61e-1f7bff11b1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:10.495532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1617c0-6edd-45b1-b61e-1f7bff11b1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:10.496520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1617c0-6edd-45b1-b61e-1f7bff11b1b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:10.497399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1617c0-6edd-45b1-b61e-1f7bff11b1b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:10.498083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1617c0-6edd-45b1-b61e-1f7bff11b1b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:10.501476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1617c0-6edd-45b1-b61e-1f7bff11b1b5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:10.518644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 701/1401] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:42:10.522984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b44df6-3373-4b5d-be3d-a9d2affe4d7a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:10.535933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b44df6-3373-4b5d-be3d-a9d2affe4d7a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:10.542064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0ecd3-ee3c-42a0-8ed7-1787c207f946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:10.544277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0ecd3-ee3c-42a0-8ed7-1787c207f946 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:10.545463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0ecd3-ee3c-42a0-8ed7-1787c207f946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:10.545966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0ecd3-ee3c-42a0-8ed7-1787c207f946 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:10.546855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0ecd3-ee3c-42a0-8ed7-1787c207f946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:10.549631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0ecd3-ee3c-42a0-8ed7-1787c207f946 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:10.570533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:10.593663 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 701/1402] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:10 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:10.597451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1617c0-6edd-45b1-b61e-1f7bff11b1b5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:10.639138 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1617c0-6edd-45b1-b61e-1f7bff11b1b5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:10.825693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be3991-9cd1-472f-9c54-fe50a4520758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:10.826629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be3991-9cd1-472f-9c54-fe50a4520758 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:10.827107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be3991-9cd1-472f-9c54-fe50a4520758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:10.827613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be3991-9cd1-472f-9c54-fe50a4520758 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:10.828438 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be3991-9cd1-472f-9c54-fe50a4520758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:10.831500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be3991-9cd1-472f-9c54-fe50a4520758 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:10.867587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 702/1403] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:10 2022] POST /identity/v3/users => generated 282 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:10.871517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0ecd3-ee3c-42a0-8ed7-1787c207f946 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:10.893943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890939b5-868b-471b-bc38-9a6b52006104 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:10.894660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890939b5-868b-471b-bc38-9a6b52006104 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:10.895125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890939b5-868b-471b-bc38-9a6b52006104 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:10.897236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 703/1404] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:42:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:42:10.903537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0ecd3-ee3c-42a0-8ed7-1787c207f946 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:10.918114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f1fad-0de5-4bfa-b18b-1ea806d90706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:10.919862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f1fad-0de5-4bfa-b18b-1ea806d90706 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:10.920112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f1fad-0de5-4bfa-b18b-1ea806d90706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:10.923433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca71694-4583-48f8-8f7a-f9f75b4dabe7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:10.924841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca71694-4583-48f8-8f7a-f9f75b4dabe7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:10.925599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca71694-4583-48f8-8f7a-f9f75b4dabe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:10.933132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f1fad-0de5-4bfa-b18b-1ea806d90706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:10.937323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f1fad-0de5-4bfa-b18b-1ea806d90706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:10.944377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f1fad-0de5-4bfa-b18b-1ea806d90706 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:10.976304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 702/1405] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:10.982850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f1fad-0de5-4bfa-b18b-1ea806d90706 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:11.002921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4ca71694-4583-48f8-8f7a-f9f75b4dabe7 None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:11.007488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f1fad-0de5-4bfa-b18b-1ea806d90706 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:11.010190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4ca71694-4583-48f8-8f7a-f9f75b4dabe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:11.040579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534c616-7579-4eee-9637-6d3352878bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:11.041403 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534c616-7579-4eee-9637-6d3352878bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:11.041881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534c616-7579-4eee-9637-6d3352878bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:11.042184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534c616-7579-4eee-9637-6d3352878bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:11.042908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534c616-7579-4eee-9637-6d3352878bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:11.044628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534c616-7579-4eee-9637-6d3352878bf1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:11.046575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:11.069142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 703/1406] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:11.075840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534c616-7579-4eee-9637-6d3352878bf1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:11.122931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534c616-7579-4eee-9637-6d3352878bf1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:11.135117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44326477-803f-4768-a055-f43546611390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:11.135802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44326477-803f-4768-a055-f43546611390 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:11.136071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44326477-803f-4768-a055-f43546611390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:11.136510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44326477-803f-4768-a055-f43546611390 admin admin] PATH_INFO: `/v3/projects/d0071bcef8e64cb98e9961b3039e6a0c/users/643f0d6822364172a0da6f0807cc0da4/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:11.202395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 704/1407] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:42:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:11.202821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44326477-803f-4768-a055-f43546611390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0071bcef8e64cb98e9961b3039e6a0c, user_id=643f0d6822364172a0da6f0807cc0da4, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:11.206684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44326477-803f-4768-a055-f43546611390 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:11.230676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 704/1408] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:11 2022] PUT /identity/v3/projects/d0071bcef8e64cb98e9961b3039e6a0c/users/643f0d6822364172a0da6f0807cc0da4/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:11.242249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5155bd-0014-4f33-bd05-e88681b3656b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:11.242611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5155bd-0014-4f33-bd05-e88681b3656b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:11.242954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5155bd-0014-4f33-bd05-e88681b3656b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:11.290958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44326477-803f-4768-a055-f43546611390 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:11.292950 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-44326477-803f-4768-a055-f43546611390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:11.320750 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9a5155bd-0014-4f33-bd05-e88681b3656b None None] MFA Rules not processed for user `643f0d6822364172a0da6f0807cc0da4`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:11.323549 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9a5155bd-0014-4f33-bd05-e88681b3656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:11.463415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44326477-803f-4768-a055-f43546611390 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:11.470924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:11.471922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:11.472207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:11.472477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] PATH_INFO: `/v3/projects/b1c671a946734eca995cd32ffe52d20b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:11.482218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 705/1409] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:11.482218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5155bd-0014-4f33-bd05-e88681b3656b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:11.482590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c671a946734eca995cd32ffe52d20b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:11.483143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:11.487498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9a5155bd-0014-4f33-bd05-e88681b3656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:11.577166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c671a946734eca995cd32ffe52d20b', 'request_id': 'req-8de5ecfe-ad0d-430e-8690-4e4095805f22'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:11.586885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c671a946734eca995cd32ffe52d20b', 'request_id': 'req-8de5ecfe-ad0d-430e-8690-4e4095805f22'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:11.599282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c671a946734eca995cd32ffe52d20b', 'request_id': 'req-8de5ecfe-ad0d-430e-8690-4e4095805f22'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:11.622044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] The token cache is being invalidate because project b1c671a946734eca995cd32ffe52d20b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:11.622292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:11.623629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:11.625626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 705/1410] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:11 2022] DELETE /identity/v3/projects/b1c671a946734eca995cd32ffe52d20b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:11.628329 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:11.633158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:11.676959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5155bd-0014-4f33-bd05-e88681b3656b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:11.738717 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:11.757145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:11.788929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cb894-90ca-43bf-8389-65b2de6c89cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:11.789925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cb894-90ca-43bf-8389-65b2de6c89cc service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:11.790479 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cb894-90ca-43bf-8389-65b2de6c89cc service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:11.791384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cb894-90ca-43bf-8389-65b2de6c89cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:11.794071 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2d2cb894-90ca-43bf-8389-65b2de6c89cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:11.815408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5ecfe-ad0d-430e-8690-4e4095805f22 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:11.816763 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cb894-90ca-43bf-8389-65b2de6c89cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:11.818768 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cb894-90ca-43bf-8389-65b2de6c89cc service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:11.827854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac79762-483d-4994-a6b7-dd78a6576ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:11.828490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac79762-483d-4994-a6b7-dd78a6576ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:11.828769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac79762-483d-4994-a6b7-dd78a6576ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:11.829035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac79762-483d-4994-a6b7-dd78a6576ba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:11.829709 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac79762-483d-4994-a6b7-dd78a6576ba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:11.834957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 706/1411] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:11.839088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cb894-90ca-43bf-8389-65b2de6c89cc service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:11.841922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2d2cb894-90ca-43bf-8389-65b2de6c89cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:11.845879 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac79762-483d-4994-a6b7-dd78a6576ba9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:11.864833 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cb894-90ca-43bf-8389-65b2de6c89cc service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:11.872722 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:11.873668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:11.874221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:11.874851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] PATH_INFO: `/v3/users/0071d5dda27f44299f86256e9e918687` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:11.876486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0071d5dda27f44299f86256e9e918687)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:11.879446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:11.926939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 706/1412] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:11 2022] POST /identity/v3/projects => generated 363 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:11.934973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac79762-483d-4994-a6b7-dd78a6576ba9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:11.973982 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0071d5dda27f44299f86256e9e918687'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:11.974716 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0071d5dda27f44299f86256e9e918687'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:11.975721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0071d5dda27f44299f86256e9e918687'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:11.984940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0071d5dda27f44299f86256e9e918687'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:12.007930 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0071d5dda27f44299f86256e9e918687'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:12.027371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 707/1413] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:11 2022] DELETE /identity/v3/users/0071d5dda27f44299f86256e9e918687 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:12.054424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac79762-483d-4994-a6b7-dd78a6576ba9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:12.176674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324f6cc-8ae2-4911-88cf-f7e3cfecefb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:12.177487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324f6cc-8ae2-4911-88cf-f7e3cfecefb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:12.178651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324f6cc-8ae2-4911-88cf-f7e3cfecefb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:12.178935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324f6cc-8ae2-4911-88cf-f7e3cfecefb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:12.179576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324f6cc-8ae2-4911-88cf-f7e3cfecefb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:12.181541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324f6cc-8ae2-4911-88cf-f7e3cfecefb0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:12.225019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 707/1414] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:11 2022] POST /identity/v3/users => generated 288 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:12.226924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:12.230419 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:12.253036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0afd29-a9f3-4448-b83c-1fd951867b0e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:12.260616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beca0d-19fd-47a2-bc7b-a4db71cd7684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:12.261505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beca0d-19fd-47a2-bc7b-a4db71cd7684 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:12.262470 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beca0d-19fd-47a2-bc7b-a4db71cd7684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:12.263001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beca0d-19fd-47a2-bc7b-a4db71cd7684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:12.263725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87beca0d-19fd-47a2-bc7b-a4db71cd7684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:12.265798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87beca0d-19fd-47a2-bc7b-a4db71cd7684 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:12.280402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 708/1415] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:42:12.285966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324f6cc-8ae2-4911-88cf-f7e3cfecefb0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:12.301365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324f6cc-8ae2-4911-88cf-f7e3cfecefb0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:12.310821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4df1c-9cbe-49d5-b4af-22377248f0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:12.311659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4df1c-9cbe-49d5-b4af-22377248f0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:12.311942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4df1c-9cbe-49d5-b4af-22377248f0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:12.312215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4df1c-9cbe-49d5-b4af-22377248f0f3 admin admin] PATH_INFO: `/v3/projects/c12755f4a61d4619b9dd5fee3cc5e695/users/c160bf1c8196472781899cc671c10298/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:12.337208 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4df1c-9cbe-49d5-b4af-22377248f0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12755f4a61d4619b9dd5fee3cc5e695, user_id=c160bf1c8196472781899cc671c10298, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:12.342303 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4df1c-9cbe-49d5-b4af-22377248f0f3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:12.374518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beca0d-19fd-47a2-bc7b-a4db71cd7684 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:12.375031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 708/1416] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:12 2022] PUT /identity/v3/projects/c12755f4a61d4619b9dd5fee3cc5e695/users/c160bf1c8196472781899cc671c10298/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:12.387993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beca0d-19fd-47a2-bc7b-a4db71cd7684 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:12.479558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b73026-47de-47c4-89db-50d106f0da47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:12.480174 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73026-47de-47c4-89db-50d106f0da47 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:12.480456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73026-47de-47c4-89db-50d106f0da47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:12.480729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73026-47de-47c4-89db-50d106f0da47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:12.481375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b73026-47de-47c4-89db-50d106f0da47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:12.483039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b73026-47de-47c4-89db-50d106f0da47 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:12.495143 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 709/1417] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:12.498901 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4df1c-9cbe-49d5-b4af-22377248f0f3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:12.510064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4df1c-9cbe-49d5-b4af-22377248f0f3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:12.523350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c0bc7-0f51-41da-8f34-ab9c4d65eacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:12.524101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c0bc7-0f51-41da-8f34-ab9c4d65eacf admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:12.524428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c0bc7-0f51-41da-8f34-ab9c4d65eacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:12.524759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c0bc7-0f51-41da-8f34-ab9c4d65eacf admin admin] PATH_INFO: `/v3/projects/c12755f4a61d4619b9dd5fee3cc5e695/users/c160bf1c8196472781899cc671c10298/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:12.527009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c0bc7-0f51-41da-8f34-ab9c4d65eacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12755f4a61d4619b9dd5fee3cc5e695, user_id=c160bf1c8196472781899cc671c10298, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:12.531494 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c0bc7-0f51-41da-8f34-ab9c4d65eacf admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:12.546239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 709/1418] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:12 2022] PUT /identity/v3/projects/c12755f4a61d4619b9dd5fee3cc5e695/users/c160bf1c8196472781899cc671c10298/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:12.557516 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd2fea-d589-468a-b0a5-77e7e45585ed None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:12.557825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd2fea-d589-468a-b0a5-77e7e45585ed None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:12.558107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd2fea-d589-468a-b0a5-77e7e45585ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:12.597715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c0bc7-0f51-41da-8f34-ab9c4d65eacf admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:12.600591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-350c0bc7-0f51-41da-8f34-ab9c4d65eacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:12.628892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-93fd2fea-d589-468a-b0a5-77e7e45585ed None None] MFA Rules not processed for user `c160bf1c8196472781899cc671c10298`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:12.630420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-93fd2fea-d589-468a-b0a5-77e7e45585ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:12.656817 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:12.692301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:12.769370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c0bc7-0f51-41da-8f34-ab9c4d65eacf admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:12.779693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:12.780343 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:12.780625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:12.780960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] PATH_INFO: `/v3/projects/b9da76fd352a402c8806f94c4e1a75cc` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:12.785292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9da76fd352a402c8806f94c4e1a75cc)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:12.787404 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:12.809238 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 710/1419] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:12.832323 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd2fea-d589-468a-b0a5-77e7e45585ed None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:12.833256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-93fd2fea-d589-468a-b0a5-77e7e45585ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:12.907022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9da76fd352a402c8806f94c4e1a75cc', 'request_id': 'req-3c46fda2-30be-4e2d-b272-7873df0eb16c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:12.918484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9da76fd352a402c8806f94c4e1a75cc', 'request_id': 'req-3c46fda2-30be-4e2d-b272-7873df0eb16c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:12.935177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9da76fd352a402c8806f94c4e1a75cc', 'request_id': 'req-3c46fda2-30be-4e2d-b272-7873df0eb16c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:12.959606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd2fea-d589-468a-b0a5-77e7e45585ed None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:12.968724 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] The token cache is being invalidate because project b9da76fd352a402c8806f94c4e1a75cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:12.969817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:12.971414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3c46fda2-30be-4e2d-b272-7873df0eb16c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:12.975121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 710/1420] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:12 2022] DELETE /identity/v3/projects/b9da76fd352a402c8806f94c4e1a75cc => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:12.980450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dfff-a079-4fa8-8dd0-5976bca414bb None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:12.982983 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dfff-a079-4fa8-8dd0-5976bca414bb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:12.985704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dfff-a079-4fa8-8dd0-5976bca414bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:13.013716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e0f0dfff-a079-4fa8-8dd0-5976bca414bb None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:13.015331 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e0f0dfff-a079-4fa8-8dd0-5976bca414bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:13.100754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866624b-9303-4100-ac51-c7f6a6d8bbee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:13.108160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866624b-9303-4100-ac51-c7f6a6d8bbee service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:13.108906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866624b-9303-4100-ac51-c7f6a6d8bbee service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:13.109335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866624b-9303-4100-ac51-c7f6a6d8bbee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:13.112826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a866624b-9303-4100-ac51-c7f6a6d8bbee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:13.138662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 711/1421] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:13.148206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba649e-1525-4ac7-b804-3b38cd272ccc None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:13.148597 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba649e-1525-4ac7-b804-3b38cd272ccc None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:13.148993 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba649e-1525-4ac7-b804-3b38cd272ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:13.214979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d8ba649e-1525-4ac7-b804-3b38cd272ccc None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:13.218546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d8ba649e-1525-4ac7-b804-3b38cd272ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:13.247316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 712/1422] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:13.250812 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba649e-1525-4ac7-b804-3b38cd272ccc None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:13.281473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba649e-1525-4ac7-b804-3b38cd272ccc None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:13.290226 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe67c3-dd12-4ded-bb3b-d45fd147b6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:13.295029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe67c3-dd12-4ded-bb3b-d45fd147b6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:13.295308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe67c3-dd12-4ded-bb3b-d45fd147b6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:13.295578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe67c3-dd12-4ded-bb3b-d45fd147b6e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:13.296256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe67c3-dd12-4ded-bb3b-d45fd147b6e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:13.296556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe67c3-dd12-4ded-bb3b-d45fd147b6e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:13.301790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe67c3-dd12-4ded-bb3b-d45fd147b6e7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:13.337085 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 713/1423] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:42:13.343313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe67c3-dd12-4ded-bb3b-d45fd147b6e7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:13.352577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866624b-9303-4100-ac51-c7f6a6d8bbee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:13.354872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866624b-9303-4100-ac51-c7f6a6d8bbee service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:13.387965 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe67c3-dd12-4ded-bb3b-d45fd147b6e7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:13.389245 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 711/1424] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:13.399645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da409e2-2f7a-40ad-95c3-076cbe3e7c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:13.400342 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da409e2-2f7a-40ad-95c3-076cbe3e7c53 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:13.400805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da409e2-2f7a-40ad-95c3-076cbe3e7c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:13.401826 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da409e2-2f7a-40ad-95c3-076cbe3e7c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:13.403261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da409e2-2f7a-40ad-95c3-076cbe3e7c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:13.411352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da409e2-2f7a-40ad-95c3-076cbe3e7c53 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:13.535260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866624b-9303-4100-ac51-c7f6a6d8bbee service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:13.537717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 714/1425] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:13 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:13.552872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866624b-9303-4100-ac51-c7f6a6d8bbee service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:13.576533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:13.589426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da409e2-2f7a-40ad-95c3-076cbe3e7c53 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:13.594595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6da409e2-2f7a-40ad-95c3-076cbe3e7c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:13.621086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:13.701187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da409e2-2f7a-40ad-95c3-076cbe3e7c53 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:13.710128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e4dab-328b-4b1d-a98c-a184db5ee9d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:13.711556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e4dab-328b-4b1d-a98c-a184db5ee9d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:13.711976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e4dab-328b-4b1d-a98c-a184db5ee9d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:13.712959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e4dab-328b-4b1d-a98c-a184db5ee9d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:13.716716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7d6e4dab-328b-4b1d-a98c-a184db5ee9d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:13.722829 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cd256-44c9-426b-ab50-c7db85684fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:13.724573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cd256-44c9-426b-ab50-c7db85684fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:13.725653 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cd256-44c9-426b-ab50-c7db85684fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:13.727474 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cd256-44c9-426b-ab50-c7db85684fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:13.728598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cd256-44c9-426b-ab50-c7db85684fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:13.732117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cd256-44c9-426b-ab50-c7db85684fd1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:13.820592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 712/1426] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:13 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:42:13.822500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cd256-44c9-426b-ab50-c7db85684fd1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:13.858322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cd256-44c9-426b-ab50-c7db85684fd1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:13.880759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96240440-e872-4acc-b0ec-b57de989c9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:13.881682 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96240440-e872-4acc-b0ec-b57de989c9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:13.883601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96240440-e872-4acc-b0ec-b57de989c9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:13.885635 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96240440-e872-4acc-b0ec-b57de989c9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:13.887309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96240440-e872-4acc-b0ec-b57de989c9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:13.889205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96240440-e872-4acc-b0ec-b57de989c9a2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:13.926691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 713/1427] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:13.937610 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96240440-e872-4acc-b0ec-b57de989c9a2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:13.953413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e4dab-328b-4b1d-a98c-a184db5ee9d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:13.956246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e4dab-328b-4b1d-a98c-a184db5ee9d1 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:13.974874 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96240440-e872-4acc-b0ec-b57de989c9a2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:13.987175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52d72d-e3b8-4307-b74a-da344fb44b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:13.987907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d72d-e3b8-4307-b74a-da344fb44b8a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:13.988287 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d72d-e3b8-4307-b74a-da344fb44b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:13.988675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d72d-e3b8-4307-b74a-da344fb44b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:13.989427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52d72d-e3b8-4307-b74a-da344fb44b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:13.991188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52d72d-e3b8-4307-b74a-da344fb44b8a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:14.014371 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 715/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:14.018712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e4dab-328b-4b1d-a98c-a184db5ee9d1 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:14.028345 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 714/1429] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:14.044826 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e4dab-328b-4b1d-a98c-a184db5ee9d1 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:14.058259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee00ea-1461-4d5c-9988-6db9b87f0224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:14.061123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee00ea-1461-4d5c-9988-6db9b87f0224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:14.061934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee00ea-1461-4d5c-9988-6db9b87f0224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:14.063218 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee00ea-1461-4d5c-9988-6db9b87f0224 admin admin] PATH_INFO: `/v3/projects/0fc6f5309ffe4d289aa931251d635749/users/a1fed6ba808a44458c71b6c6a1d4bef6/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:14.111310 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee00ea-1461-4d5c-9988-6db9b87f0224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc6f5309ffe4d289aa931251d635749, user_id=a1fed6ba808a44458c71b6c6a1d4bef6, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:14.115415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee00ea-1461-4d5c-9988-6db9b87f0224 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:14.193268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 716/1430] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:14 2022] PUT /identity/v3/projects/0fc6f5309ffe4d289aa931251d635749/users/a1fed6ba808a44458c71b6c6a1d4bef6/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:14.205237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b2218-ecaf-464a-8677-29f7e8013b29 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:14.205523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b2218-ecaf-464a-8677-29f7e8013b29 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:14.205798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b2218-ecaf-464a-8677-29f7e8013b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:14.307472 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f97b2218-ecaf-464a-8677-29f7e8013b29 None None] MFA Rules not processed for user `a1fed6ba808a44458c71b6c6a1d4bef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:14.308708 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f97b2218-ecaf-464a-8677-29f7e8013b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:14.606772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 715/1431] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:14.610660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee00ea-1461-4d5c-9988-6db9b87f0224 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:14.822926 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee00ea-1461-4d5c-9988-6db9b87f0224 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:14.835629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ecc583-f71f-48b7-8d27-475d289b0f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:14.838580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecc583-f71f-48b7-8d27-475d289b0f23 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:14.839070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecc583-f71f-48b7-8d27-475d289b0f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:14.839514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ecc583-f71f-48b7-8d27-475d289b0f23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:14.840381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ecc583-f71f-48b7-8d27-475d289b0f23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:14.845352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ecc583-f71f-48b7-8d27-475d289b0f23 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:14.977160 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 717/1432] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:14 2022] POST /identity/v3/projects => generated 385 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:14.979330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b2218-ecaf-464a-8677-29f7e8013b29 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:15.003688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b2218-ecaf-464a-8677-29f7e8013b29 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:15.193632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153f7d1-89f6-4dc5-aa4b-bab49614dd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:15.194110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153f7d1-89f6-4dc5-aa4b-bab49614dd54 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:15.196051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153f7d1-89f6-4dc5-aa4b-bab49614dd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:15.196726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153f7d1-89f6-4dc5-aa4b-bab49614dd54 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:15.197407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153f7d1-89f6-4dc5-aa4b-bab49614dd54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:15.214039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153f7d1-89f6-4dc5-aa4b-bab49614dd54 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:15.240375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:15.250964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 716/1433] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:14 2022] POST /identity/v3/users => generated 299 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:15.252365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ecc583-f71f-48b7-8d27-475d289b0f23 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:15.266124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ecc583-f71f-48b7-8d27-475d289b0f23 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:15.275321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638be8-a986-4f87-8f87-6d726b4acc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:15.276647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638be8-a986-4f87-8f87-6d726b4acc9e admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:15.277040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638be8-a986-4f87-8f87-6d726b4acc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:15.277391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638be8-a986-4f87-8f87-6d726b4acc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:15.278376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4638be8-a986-4f87-8f87-6d726b4acc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:15.280074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4638be8-a986-4f87-8f87-6d726b4acc9e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:15.300468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153f7d1-89f6-4dc5-aa4b-bab49614dd54 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:15.310029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 718/1434] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:15.322018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153f7d1-89f6-4dc5-aa4b-bab49614dd54 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:15.338312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86eda14-ca0b-4a63-be75-6e03f02205a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:15.340772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eda14-ca0b-4a63-be75-6e03f02205a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:15.341164 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eda14-ca0b-4a63-be75-6e03f02205a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:15.346408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eda14-ca0b-4a63-be75-6e03f02205a7 admin admin] PATH_INFO: `/v3/projects/3e3e1c262a434cc39dadd6a4dd3324a3/users/470f47b2493743da97a9a1884eb2aae9/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:15.373832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638be8-a986-4f87-8f87-6d726b4acc9e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:15.375675 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c4638be8-a986-4f87-8f87-6d726b4acc9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:15.386862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86eda14-ca0b-4a63-be75-6e03f02205a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3e1c262a434cc39dadd6a4dd3324a3, user_id=470f47b2493743da97a9a1884eb2aae9, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:15.393790 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86eda14-ca0b-4a63-be75-6e03f02205a7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:15.411527 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638be8-a986-4f87-8f87-6d726b4acc9e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:15.426932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa9bf0-1781-4bc3-8b16-f1c80743b305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:15.432584 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa9bf0-1781-4bc3-8b16-f1c80743b305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:15.432934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa9bf0-1781-4bc3-8b16-f1c80743b305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:15.433202 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa9bf0-1781-4bc3-8b16-f1c80743b305 admin admin] PATH_INFO: `/v3/users/1bcd5392ff7a49908d9140fefea19ca0` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:15.434334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 717/1435] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:15 2022] PUT /identity/v3/projects/3e3e1c262a434cc39dadd6a4dd3324a3/users/470f47b2493743da97a9a1884eb2aae9/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:15.434740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa9bf0-1781-4bc3-8b16-f1c80743b305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bcd5392ff7a49908d9140fefea19ca0)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:15.436254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86eda14-ca0b-4a63-be75-6e03f02205a7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:15.436846 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa9bf0-1781-4bc3-8b16-f1c80743b305 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:15.460397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:15.594132 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86eda14-ca0b-4a63-be75-6e03f02205a7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:15.598644 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1baa9bf0-1781-4bc3-8b16-f1c80743b305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bcd5392ff7a49908d9140fefea19ca0'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:15.600619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1baa9bf0-1781-4bc3-8b16-f1c80743b305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bcd5392ff7a49908d9140fefea19ca0'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:15.623269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93204c88-3586-4214-b715-b428e1c53cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:15.624065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93204c88-3586-4214-b715-b428e1c53cea admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:15.625040 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93204c88-3586-4214-b715-b428e1c53cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:15.625961 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93204c88-3586-4214-b715-b428e1c53cea admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:15.628304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93204c88-3586-4214-b715-b428e1c53cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:15.635113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93204c88-3586-4214-b715-b428e1c53cea admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:15.636301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1baa9bf0-1781-4bc3-8b16-f1c80743b305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bcd5392ff7a49908d9140fefea19ca0'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:15.638024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1baa9bf0-1781-4bc3-8b16-f1c80743b305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bcd5392ff7a49908d9140fefea19ca0'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:15.662256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 718/1436] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:15.662838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-1baa9bf0-1781-4bc3-8b16-f1c80743b305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bcd5392ff7a49908d9140fefea19ca0'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:15.663456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93204c88-3586-4214-b715-b428e1c53cea admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:15.668875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-93204c88-3586-4214-b715-b428e1c53cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:15.708069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93204c88-3586-4214-b715-b428e1c53cea admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:15.709135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 719/1437] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:15 2022] DELETE /identity/v3/users/1bcd5392ff7a49908d9140fefea19ca0 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:15.713944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6fef7-57ce-4d51-a6e2-65abe7664dab None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:15.714351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6fef7-57ce-4d51-a6e2-65abe7664dab None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:15.714730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6fef7-57ce-4d51-a6e2-65abe7664dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:15.798820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-cdb6fef7-57ce-4d51-a6e2-65abe7664dab None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:15.803116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cdb6fef7-57ce-4d51-a6e2-65abe7664dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:15.859285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2150c2d-cf9f-4396-80ee-52b14141eb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:15.860142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2150c2d-cf9f-4396-80ee-52b14141eb81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:15.860511 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2150c2d-cf9f-4396-80ee-52b14141eb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:15.860711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2150c2d-cf9f-4396-80ee-52b14141eb81 admin admin] PATH_INFO: `/v3/projects/3e3e1c262a434cc39dadd6a4dd3324a3/users/470f47b2493743da97a9a1884eb2aae9/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:15.867804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2150c2d-cf9f-4396-80ee-52b14141eb81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3e1c262a434cc39dadd6a4dd3324a3, user_id=470f47b2493743da97a9a1884eb2aae9, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:15.870404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2150c2d-cf9f-4396-80ee-52b14141eb81 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:15.910345 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 719/1438] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:15 2022] PUT /identity/v3/projects/3e3e1c262a434cc39dadd6a4dd3324a3/users/470f47b2493743da97a9a1884eb2aae9/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:15.916725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed664365-bcd2-44f3-881e-7517e1e65a8c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:15.916725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed664365-bcd2-44f3-881e-7517e1e65a8c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:15.916725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed664365-bcd2-44f3-881e-7517e1e65a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:15.978089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ed664365-bcd2-44f3-881e-7517e1e65a8c None None] MFA Rules not processed for user `470f47b2493743da97a9a1884eb2aae9`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:15.979402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ed664365-bcd2-44f3-881e-7517e1e65a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:15.998367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 720/1439] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:16.105538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:16.146802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 720/1440] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:16.164587 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6fef7-57ce-4d51-a6e2-65abe7664dab None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:16.166218 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cdb6fef7-57ce-4d51-a6e2-65abe7664dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:16.185398 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed664365-bcd2-44f3-881e-7517e1e65a8c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:16.186959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ed664365-bcd2-44f3-881e-7517e1e65a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:16.416870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6fef7-57ce-4d51-a6e2-65abe7664dab None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:16.517599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed664365-bcd2-44f3-881e-7517e1e65a8c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:16.517957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:16.525135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd652b8f-5f11-41af-8c09-1d1af240d3af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:16.525780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652b8f-5f11-41af-8c09-1d1af240d3af service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:16.526126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652b8f-5f11-41af-8c09-1d1af240d3af service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:16.526414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd652b8f-5f11-41af-8c09-1d1af240d3af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:16.527891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaef89-2a93-42bd-ad93-750b2d02ef05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:16.528797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaef89-2a93-42bd-ad93-750b2d02ef05 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:16.529379 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaef89-2a93-42bd-ad93-750b2d02ef05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:16.529674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaef89-2a93-42bd-ad93-750b2d02ef05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:16.566242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aaef89-2a93-42bd-ad93-750b2d02ef05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:16.568645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aaef89-2a93-42bd-ad93-750b2d02ef05 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:16.596407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 721/1441] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:16.706246 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd652b8f-5f11-41af-8c09-1d1af240d3af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:16.707631 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd652b8f-5f11-41af-8c09-1d1af240d3af service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:16.730306 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 721/1442] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:16.754721 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaef89-2a93-42bd-ad93-750b2d02ef05 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:16.773920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaef89-2a93-42bd-ad93-750b2d02ef05 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:16.786000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:16.786862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:16.790445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:16.790815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] PATH_INFO: `/v3/users/a1fed6ba808a44458c71b6c6a1d4bef6` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:16.793892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1fed6ba808a44458c71b6c6a1d4bef6)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:16.799858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:16.939521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1fed6ba808a44458c71b6c6a1d4bef6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:16.941609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1fed6ba808a44458c71b6c6a1d4bef6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:16.975462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1fed6ba808a44458c71b6c6a1d4bef6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:16.978275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1fed6ba808a44458c71b6c6a1d4bef6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:17.007441 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1fed6ba808a44458c71b6c6a1d4bef6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:17.052313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 722/1443] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:16 2022] DELETE /identity/v3/users/a1fed6ba808a44458c71b6c6a1d4bef6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:17.056990 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c79bbb-7435-4688-b8f4-7d4afe4448e6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:17.057564 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c79bbb-7435-4688-b8f4-7d4afe4448e6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:17.058062 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c79bbb-7435-4688-b8f4-7d4afe4448e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:17.099357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-77c79bbb-7435-4688-b8f4-7d4afe4448e6 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:17.101760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-77c79bbb-7435-4688-b8f4-7d4afe4448e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:17.227742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 722/1444] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:17.251382 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:17.253253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:17.383381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a789ec-2b4e-4d96-8158-cc8121dcffd9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:17.398226 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c1a6f-76fc-4524-8776-0f3a9a8799f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:17.399222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c1a6f-76fc-4524-8776-0f3a9a8799f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:17.399647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c1a6f-76fc-4524-8776-0f3a9a8799f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:17.400086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c1a6f-76fc-4524-8776-0f3a9a8799f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:17.436408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c1a6f-76fc-4524-8776-0f3a9a8799f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:17.437945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c1a6f-76fc-4524-8776-0f3a9a8799f6 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:17.483620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 723/1445] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:18.108876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c79bbb-7435-4688-b8f4-7d4afe4448e6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:18.111663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-77c79bbb-7435-4688-b8f4-7d4afe4448e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:18.146937 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c79bbb-7435-4688-b8f4-7d4afe4448e6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:18.178390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:18.178390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:18.178390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:18.178390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] PATH_INFO: `/v3/users/643f0d6822364172a0da6f0807cc0da4` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:18.178812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=643f0d6822364172a0da6f0807cc0da4)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:18.178812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:18.250209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:18.264972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643f0d6822364172a0da6f0807cc0da4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:18.265757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643f0d6822364172a0da6f0807cc0da4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:18.266726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643f0d6822364172a0da6f0807cc0da4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:18.277243 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643f0d6822364172a0da6f0807cc0da4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:18.296027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643f0d6822364172a0da6f0807cc0da4'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:18.336694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 723/1446] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:18 2022] DELETE /identity/v3/users/643f0d6822364172a0da6f0807cc0da4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:18.344387 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9b68-ca91-43f1-96a8-de88fdacd02f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:18.345339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9b68-ca91-43f1-96a8-de88fdacd02f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:18.346040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9b68-ca91-43f1-96a8-de88fdacd02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:18.386145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-baac9b68-ca91-43f1-96a8-de88fdacd02f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:18.386145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-baac9b68-ca91-43f1-96a8-de88fdacd02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:18.561835 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 724/1447] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:18.615564 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:18.617508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:18.701257 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138b52b-ddd5-49a6-950c-0d3ed748f373 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:18.706059 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963820f-4fdb-48c6-abd4-de77db3b6903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:18.706891 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963820f-4fdb-48c6-abd4-de77db3b6903 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:18.707181 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963820f-4fdb-48c6-abd4-de77db3b6903 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:18.707449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963820f-4fdb-48c6-abd4-de77db3b6903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:18.743883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963820f-4fdb-48c6-abd4-de77db3b6903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:18.745396 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963820f-4fdb-48c6-abd4-de77db3b6903 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:18.776862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 724/1448] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:19.445102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac9b68-ca91-43f1-96a8-de88fdacd02f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:19.446537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-baac9b68-ca91-43f1-96a8-de88fdacd02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:19.457060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:19.473410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac9b68-ca91-43f1-96a8-de88fdacd02f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:19.484793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:19.485865 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:19.486455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:19.486923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] PATH_INFO: `/v3/projects/96dac3b390514fabb62cb2d86ef16d97` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:19.488429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96dac3b390514fabb62cb2d86ef16d97)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:19.490566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:19.561795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96dac3b390514fabb62cb2d86ef16d97', 'request_id': 'req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:19.581336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96dac3b390514fabb62cb2d86ef16d97', 'request_id': 'req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:19.607649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96dac3b390514fabb62cb2d86ef16d97', 'request_id': 'req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:19.628148 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] The token cache is being invalidate because project 96dac3b390514fabb62cb2d86ef16d97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:19.628578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:19.629220 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:19.635512 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963820f-4fdb-48c6-abd4-de77db3b6903 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:19.639027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 725/1449] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:19 2022] DELETE /identity/v3/projects/96dac3b390514fabb62cb2d86ef16d97 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:19.671603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963820f-4fdb-48c6-abd4-de77db3b6903 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:19.759484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:19.760035 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:19.760359 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:19.760581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] PATH_INFO: `/v3/users/64f259a6b39f4df7b52f6c59e8657c3a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:19.761822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64f259a6b39f4df7b52f6c59e8657c3a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:19.765794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:19.851235 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64f259a6b39f4df7b52f6c59e8657c3a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:19.852171 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64f259a6b39f4df7b52f6c59e8657c3a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:19.853284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64f259a6b39f4df7b52f6c59e8657c3a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:19.865015 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64f259a6b39f4df7b52f6c59e8657c3a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:19.870446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:19.894003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64f259a6b39f4df7b52f6c59e8657c3a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:19.927501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 725/1450] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:19 2022] DELETE /identity/v3/users/64f259a6b39f4df7b52f6c59e8657c3a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:20.389158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:20.390815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:20.420402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595b6db-3d4c-4e08-ade2-2fe17b9b6628 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:20.559256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:20.560067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:20.560349 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:20.560622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] PATH_INFO: `/v3/projects/0fc6f5309ffe4d289aa931251d635749` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:20.561935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fc6f5309ffe4d289aa931251d635749)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:20.564669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:20.614908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fc6f5309ffe4d289aa931251d635749', 'request_id': 'req-3513615b-3464-48d9-b55d-c4c0a9580da3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:20.620504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fc6f5309ffe4d289aa931251d635749', 'request_id': 'req-3513615b-3464-48d9-b55d-c4c0a9580da3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:20.629181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fc6f5309ffe4d289aa931251d635749', 'request_id': 'req-3513615b-3464-48d9-b55d-c4c0a9580da3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:20.642770 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] The token cache is being invalidate because project 0fc6f5309ffe4d289aa931251d635749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:20.643066 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:20.643904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3513615b-3464-48d9-b55d-c4c0a9580da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:20.645758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 726/1451] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:20 2022] DELETE /identity/v3/projects/0fc6f5309ffe4d289aa931251d635749 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:20.649596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:20.651074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:20.668933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aa6f3-f92d-474d-a43b-e4d02d7f2210 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:20.741848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fcfcf-1c6c-44d5-ab46-12520b2e0809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:20.742524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fcfcf-1c6c-44d5-ab46-12520b2e0809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:20.742801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fcfcf-1c6c-44d5-ab46-12520b2e0809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:20.743066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fcfcf-1c6c-44d5-ab46-12520b2e0809 admin admin] PATH_INFO: `/v3/users/470f47b2493743da97a9a1884eb2aae9` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:20.744203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fcfcf-1c6c-44d5-ab46-12520b2e0809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=470f47b2493743da97a9a1884eb2aae9)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:20.746106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fcfcf-1c6c-44d5-ab46-12520b2e0809 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:20.805151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-141fcfcf-1c6c-44d5-ab46-12520b2e0809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '470f47b2493743da97a9a1884eb2aae9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:20.806092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-141fcfcf-1c6c-44d5-ab46-12520b2e0809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '470f47b2493743da97a9a1884eb2aae9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:20.806758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-141fcfcf-1c6c-44d5-ab46-12520b2e0809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '470f47b2493743da97a9a1884eb2aae9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:20.815674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-141fcfcf-1c6c-44d5-ab46-12520b2e0809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '470f47b2493743da97a9a1884eb2aae9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:20.828150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-141fcfcf-1c6c-44d5-ab46-12520b2e0809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '470f47b2493743da97a9a1884eb2aae9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:20.842786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 726/1452] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:20 2022] DELETE /identity/v3/users/470f47b2493743da97a9a1884eb2aae9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:20.949565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4f4a5-f243-4a86-9910-940b09f3cbcd None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:20.950253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4f4a5-f243-4a86-9910-940b09f3cbcd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:20.950862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4f4a5-f243-4a86-9910-940b09f3cbcd None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:20.952841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 727/1453] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:42:21.014608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74c447-2171-44fd-8242-e8bc5c0ccf13 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:21.014899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74c447-2171-44fd-8242-e8bc5c0ccf13 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:21.015189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74c447-2171-44fd-8242-e8bc5c0ccf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:21.070943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-dc74c447-2171-44fd-8242-e8bc5c0ccf13 None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:21.072419 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-dc74c447-2171-44fd-8242-e8bc5c0ccf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:21.157421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:21.195452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 727/1454] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:42:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:21.222465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4f4a5-f243-4a86-9910-940b09f3cbcd None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:21.225300 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fee4f4a5-f243-4a86-9910-940b09f3cbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:21.249771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74c447-2171-44fd-8242-e8bc5c0ccf13 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:21.252326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-dc74c447-2171-44fd-8242-e8bc5c0ccf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:21.266283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:21.272133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74c447-2171-44fd-8242-e8bc5c0ccf13 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:21.291121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc622a5-ce79-4256-82b4-eea432763d66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:21.294319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc622a5-ce79-4256-82b4-eea432763d66 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:21.294964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc622a5-ce79-4256-82b4-eea432763d66 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:21.295238 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc622a5-ce79-4256-82b4-eea432763d66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:21.323939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc622a5-ce79-4256-82b4-eea432763d66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:21.325710 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc622a5-ce79-4256-82b4-eea432763d66 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:21.354821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4f4a5-f243-4a86-9910-940b09f3cbcd None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:21.359873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:21.360645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:21.361556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:21.361835 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] PATH_INFO: `/v3/projects/3e3e1c262a434cc39dadd6a4dd3324a3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:21.363270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e3e1c262a434cc39dadd6a4dd3324a3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:21.368010 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:21.376020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 728/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:42:21.440518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e3e1c262a434cc39dadd6a4dd3324a3', 'request_id': 'req-cc10d28a-275e-4e98-8362-b4ea6565debf'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.448564 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e3e1c262a434cc39dadd6a4dd3324a3', 'request_id': 'req-cc10d28a-275e-4e98-8362-b4ea6565debf'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.465252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e3e1c262a434cc39dadd6a4dd3324a3', 'request_id': 'req-cc10d28a-275e-4e98-8362-b4ea6565debf'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.490784 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] The token cache is being invalidate because project 3e3e1c262a434cc39dadd6a4dd3324a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:21.491302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.492300 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.499267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 728/1456] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:21 2022] DELETE /identity/v3/projects/3e3e1c262a434cc39dadd6a4dd3324a3 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:21.500998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5968-d1bb-426e-a546-180fe91fd8e9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:21.500998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5968-d1bb-426e-a546-180fe91fd8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:21.500998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5968-d1bb-426e-a546-180fe91fd8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:21.531080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:21.531982 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d48a5968-d1bb-426e-a546-180fe91fd8e9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:21.534627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:21.535548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d48a5968-d1bb-426e-a546-180fe91fd8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:21.690767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10d28a-275e-4e98-8362-b4ea6565debf admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:21.699679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:21.700467 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:21.700598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:21.700868 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] PATH_INFO: `/v3/projects/d0071bcef8e64cb98e9961b3039e6a0c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:21.703531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0071bcef8e64cb98e9961b3039e6a0c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:21.706267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:21.716597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 729/1457] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:21.719531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a5968-d1bb-426e-a546-180fe91fd8e9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:21.723919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d48a5968-d1bb-426e-a546-180fe91fd8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:21.756430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a5968-d1bb-426e-a546-180fe91fd8e9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:21.769637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:21.770348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:21.770628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:21.770938 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] PATH_INFO: `/v3/projects/d4373f323c6345558ce0e9d1195f2776` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:21.772141 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4373f323c6345558ce0e9d1195f2776)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:21.777673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:21.814627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0071bcef8e64cb98e9961b3039e6a0c', 'request_id': 'req-60d74a95-a18e-438c-884f-b481bd22e7bb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.815865 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:21.823331 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0071bcef8e64cb98e9961b3039e6a0c', 'request_id': 'req-60d74a95-a18e-438c-884f-b481bd22e7bb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.836942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0071bcef8e64cb98e9961b3039e6a0c', 'request_id': 'req-60d74a95-a18e-438c-884f-b481bd22e7bb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.853398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] The token cache is being invalidate because project d0071bcef8e64cb98e9961b3039e6a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:21.853686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.854510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-60d74a95-a18e-438c-884f-b481bd22e7bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.863617 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 729/1458] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:21 2022] DELETE /identity/v3/projects/d0071bcef8e64cb98e9961b3039e6a0c => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:21.872497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4373f323c6345558ce0e9d1195f2776', 'request_id': 'req-1d039e7b-3807-4d7a-a552-ee27b05b4d72'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.872628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f59a00-4860-44cd-9f3c-bc109908a9e5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:21.872906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f59a00-4860-44cd-9f3c-bc109908a9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:21.873178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f59a00-4860-44cd-9f3c-bc109908a9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:21.894220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4373f323c6345558ce0e9d1195f2776', 'request_id': 'req-1d039e7b-3807-4d7a-a552-ee27b05b4d72'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.895623 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b0f59a00-4860-44cd-9f3c-bc109908a9e5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:21.896839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b0f59a00-4860-44cd-9f3c-bc109908a9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:21.904880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4373f323c6345558ce0e9d1195f2776', 'request_id': 'req-1d039e7b-3807-4d7a-a552-ee27b05b4d72'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.940275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] The token cache is being invalidate because project d4373f323c6345558ce0e9d1195f2776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:21.940574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.942346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:21.946956 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 730/1459] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:21 2022] DELETE /identity/v3/projects/d4373f323c6345558ce0e9d1195f2776 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:21.946956 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:21.947151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:21.970880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 730/1460] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:21.972315 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59131aeb-fd75-4a4d-87f8-0c77cbd2091d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:21.972595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59131aeb-fd75-4a4d-87f8-0c77cbd2091d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:21.972893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59131aeb-fd75-4a4d-87f8-0c77cbd2091d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:21.993984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-59131aeb-fd75-4a4d-87f8-0c77cbd2091d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:21.995199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-59131aeb-fd75-4a4d-87f8-0c77cbd2091d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:22.004736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 731/1461] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:22.005792 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59131aeb-fd75-4a4d-87f8-0c77cbd2091d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.020962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59131aeb-fd75-4a4d-87f8-0c77cbd2091d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.025699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b16cba-e49a-4af6-afbc-925eabd3939d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.026397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b16cba-e49a-4af6-afbc-925eabd3939d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.026743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b16cba-e49a-4af6-afbc-925eabd3939d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.027085 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b16cba-e49a-4af6-afbc-925eabd3939d admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.027813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b16cba-e49a-4af6-afbc-925eabd3939d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:22.028175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b16cba-e49a-4af6-afbc-925eabd3939d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.028763 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:22.029825 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b16cba-e49a-4af6-afbc-925eabd3939d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.032944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d039e7b-3807-4d7a-a552-ee27b05b4d72 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.039589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.040435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.040725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.041240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] PATH_INFO: `/v3/users/c160bf1c8196472781899cc671c10298` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.042159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 732/1462] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:42:22.042977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c160bf1c8196472781899cc671c10298)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.044923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.046732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92321923-7bd5-4884-857d-05ab137cd18f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.047016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92321923-7bd5-4884-857d-05ab137cd18f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.047292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92321923-7bd5-4884-857d-05ab137cd18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.068381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-92321923-7bd5-4884-857d-05ab137cd18f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:22.069615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-92321923-7bd5-4884-857d-05ab137cd18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:22.078447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 733/1463] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:22.079209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92321923-7bd5-4884-857d-05ab137cd18f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.089374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92321923-7bd5-4884-857d-05ab137cd18f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.094084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191ce3a-8a21-45cc-8ecb-3cb189ef73f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.094699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191ce3a-8a21-45cc-8ecb-3cb189ef73f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.094978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191ce3a-8a21-45cc-8ecb-3cb189ef73f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.095780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191ce3a-8a21-45cc-8ecb-3cb189ef73f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.096750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191ce3a-8a21-45cc-8ecb-3cb189ef73f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.098338 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c160bf1c8196472781899cc671c10298'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:22.098654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191ce3a-8a21-45cc-8ecb-3cb189ef73f3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.098999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c160bf1c8196472781899cc671c10298'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:22.099638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c160bf1c8196472781899cc671c10298'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:22.106842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c160bf1c8196472781899cc671c10298'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:22.120758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c160bf1c8196472781899cc671c10298'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:22.128717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 734/1464] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:22 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:22.129751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191ce3a-8a21-45cc-8ecb-3cb189ef73f3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.131509 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3191ce3a-8a21-45cc-8ecb-3cb189ef73f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:22.134612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 731/1465] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:21 2022] DELETE /identity/v3/users/c160bf1c8196472781899cc671c10298 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:22.135491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.137721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:22.212394 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191ce3a-8a21-45cc-8ecb-3cb189ef73f3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.217970 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc17bf-71b1-4f4e-b210-f81f13c6a760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.218784 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc17bf-71b1-4f4e-b210-f81f13c6a760 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.219059 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc17bf-71b1-4f4e-b210-f81f13c6a760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.219381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc17bf-71b1-4f4e-b210-f81f13c6a760 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.221109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc17bf-71b1-4f4e-b210-f81f13c6a760 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:22.221494 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc17bf-71b1-4f4e-b210-f81f13c6a760 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.223340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc17bf-71b1-4f4e-b210-f81f13c6a760 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.224137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e694821-acd4-4f80-9797-183bd387bcf9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.231384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7154d0-1161-4915-8469-069762c10e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.232787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7154d0-1161-4915-8469-069762c10e96 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.233603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7154d0-1161-4915-8469-069762c10e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.233898 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7154d0-1161-4915-8469-069762c10e96 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.234686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7154d0-1161-4915-8469-069762c10e96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.237672 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7154d0-1161-4915-8469-069762c10e96 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.238577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 735/1466] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:42:22.242662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc17bf-71b1-4f4e-b210-f81f13c6a760 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.261203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc17bf-71b1-4f4e-b210-f81f13c6a760 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.269013 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e446e25-f754-4fe6-bba7-f2b9ddc4dd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.269856 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e446e25-f754-4fe6-bba7-f2b9ddc4dd4a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.270268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e446e25-f754-4fe6-bba7-f2b9ddc4dd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.270691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e446e25-f754-4fe6-bba7-f2b9ddc4dd4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.271977 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e446e25-f754-4fe6-bba7-f2b9ddc4dd4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.275006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e446e25-f754-4fe6-bba7-f2b9ddc4dd4a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.279407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 732/1467] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:22 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:22.280990 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7154d0-1161-4915-8469-069762c10e96 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.294620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7154d0-1161-4915-8469-069762c10e96 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.300834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c9296-0db3-4ab3-bcba-f6d55f5eedaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.301451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c9296-0db3-4ab3-bcba-f6d55f5eedaf admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.301732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c9296-0db3-4ab3-bcba-f6d55f5eedaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.302005 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c9296-0db3-4ab3-bcba-f6d55f5eedaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.303053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c9296-0db3-4ab3-bcba-f6d55f5eedaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.305631 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c9296-0db3-4ab3-bcba-f6d55f5eedaf admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.317147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 733/1468] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:22.318633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 736/1469] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:22 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:42:22.323147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e446e25-f754-4fe6-bba7-f2b9ddc4dd4a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.323298 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c9296-0db3-4ab3-bcba-f6d55f5eedaf admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.329302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:22.334612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c9296-0db3-4ab3-bcba-f6d55f5eedaf admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.421716 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:22.431854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542692b-5c27-4365-a3a9-3d44b3478e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.432892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542692b-5c27-4365-a3a9-3d44b3478e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.434279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542692b-5c27-4365-a3a9-3d44b3478e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.434692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542692b-5c27-4365-a3a9-3d44b3478e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.435909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542692b-5c27-4365-a3a9-3d44b3478e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.439213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542692b-5c27-4365-a3a9-3d44b3478e4a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.459183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 734/1470] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:22.459474 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e446e25-f754-4fe6-bba7-f2b9ddc4dd4a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.460764 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542692b-5c27-4365-a3a9-3d44b3478e4a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.463377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cd89c-4fec-47f6-9078-e7d774aeafb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.464821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cd89c-4fec-47f6-9078-e7d774aeafb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.465387 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cd89c-4fec-47f6-9078-e7d774aeafb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.465745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cd89c-4fec-47f6-9078-e7d774aeafb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.466522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cd89c-4fec-47f6-9078-e7d774aeafb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.468770 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cd89c-4fec-47f6-9078-e7d774aeafb0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.477734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542692b-5c27-4365-a3a9-3d44b3478e4a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.484708 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc77f6-a207-47bf-a97f-cfa3eef5f852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.485405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc77f6-a207-47bf-a97f-cfa3eef5f852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.485751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc77f6-a207-47bf-a97f-cfa3eef5f852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.486611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc77f6-a207-47bf-a97f-cfa3eef5f852 admin admin] PATH_INFO: `/v3/projects/d96389473d9f4fcab128adc205986054/users/de8b7fba46ee439fad655e504feb12fc/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.500809 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 737/1471] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:22 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:22.504612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cd89c-4fec-47f6-9078-e7d774aeafb0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.507848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc77f6-a207-47bf-a97f-cfa3eef5f852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d96389473d9f4fcab128adc205986054, user_id=de8b7fba46ee439fad655e504feb12fc, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.512528 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc77f6-a207-47bf-a97f-cfa3eef5f852 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.520363 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cd89c-4fec-47f6-9078-e7d774aeafb0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.525969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e996796-4029-408c-a253-af61383eb8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.526682 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e996796-4029-408c-a253-af61383eb8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.527042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e996796-4029-408c-a253-af61383eb8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.527318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e996796-4029-408c-a253-af61383eb8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.528006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e996796-4029-408c-a253-af61383eb8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.529882 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e996796-4029-408c-a253-af61383eb8e9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.542671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 735/1472] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:22 2022] PUT /identity/v3/projects/d96389473d9f4fcab128adc205986054/users/de8b7fba46ee439fad655e504feb12fc/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:22.550277 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc77f6-a207-47bf-a97f-cfa3eef5f852 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.550459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 738/1473] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:22.553024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d57896-1910-47ce-8145-814fc34263fa None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.553434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d57896-1910-47ce-8145-814fc34263fa None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.553779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d57896-1910-47ce-8145-814fc34263fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.611941 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b0d57896-1910-47ce-8145-814fc34263fa None None] MFA Rules not processed for user `de8b7fba46ee439fad655e504feb12fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:22.613518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b0d57896-1910-47ce-8145-814fc34263fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:22.687388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc77f6-a207-47bf-a97f-cfa3eef5f852 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.697478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d118d54-e997-481c-8bd9-48886f679359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.698568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d118d54-e997-481c-8bd9-48886f679359 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.699033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d118d54-e997-481c-8bd9-48886f679359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.699557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d118d54-e997-481c-8bd9-48886f679359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.700720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d118d54-e997-481c-8bd9-48886f679359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.703561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d118d54-e997-481c-8bd9-48886f679359 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.717776 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 736/1474] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:42:22.722805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d118d54-e997-481c-8bd9-48886f679359 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.755713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 739/1475] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:22.756209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d118d54-e997-481c-8bd9-48886f679359 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.769499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d57896-1910-47ce-8145-814fc34263fa None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.769704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de4076-bf10-4b46-8ab0-e796bff3a0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.770424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de4076-bf10-4b46-8ab0-e796bff3a0ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.770767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de4076-bf10-4b46-8ab0-e796bff3a0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.771052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de4076-bf10-4b46-8ab0-e796bff3a0ce admin admin] PATH_INFO: `/v3/projects/21a9da389eb640a89e9157992ef607ab/users/4c73b1acf31d4ef48368c9fc8c0890bc/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.803798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d57896-1910-47ce-8145-814fc34263fa None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:22.809323 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de4076-bf10-4b46-8ab0-e796bff3a0ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a9da389eb640a89e9157992ef607ab, user_id=4c73b1acf31d4ef48368c9fc8c0890bc, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.812221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de4076-bf10-4b46-8ab0-e796bff3a0ce admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.815557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38608df6-7728-4964-a693-ffcc139375ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:22.816200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38608df6-7728-4964-a693-ffcc139375ae admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.816639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38608df6-7728-4964-a693-ffcc139375ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.816916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38608df6-7728-4964-a693-ffcc139375ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.817903 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38608df6-7728-4964-a693-ffcc139375ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:22.821092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38608df6-7728-4964-a693-ffcc139375ae admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:22.853472 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 737/1476] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:22 2022] PUT /identity/v3/projects/21a9da389eb640a89e9157992ef607ab/users/4c73b1acf31d4ef48368c9fc8c0890bc/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:22.855911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c631872-d3b5-4d75-b42f-cbddec6fc40c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:22.855911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c631872-d3b5-4d75-b42f-cbddec6fc40c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:22.855911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c631872-d3b5-4d75-b42f-cbddec6fc40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:22.904002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 740/1477] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:22 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:22.904525 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2c631872-d3b5-4d75-b42f-cbddec6fc40c None None] MFA Rules not processed for user `4c73b1acf31d4ef48368c9fc8c0890bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:22.905686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38608df6-7728-4964-a693-ffcc139375ae admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:22.905801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2c631872-d3b5-4d75-b42f-cbddec6fc40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:23.030401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:23.051720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 738/1478] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:23.052837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38608df6-7728-4964-a693-ffcc139375ae admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.056484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c631872-d3b5-4d75-b42f-cbddec6fc40c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.059774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c17ea6-de93-44fb-8789-9a95aa03d976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.060648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c17ea6-de93-44fb-8789-9a95aa03d976 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.060934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c17ea6-de93-44fb-8789-9a95aa03d976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.061206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c17ea6-de93-44fb-8789-9a95aa03d976 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.061851 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c17ea6-de93-44fb-8789-9a95aa03d976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.063832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c17ea6-de93-44fb-8789-9a95aa03d976 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.081056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c631872-d3b5-4d75-b42f-cbddec6fc40c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.087676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba14dfe-8d2c-4187-96c2-794b2107afeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.088370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba14dfe-8d2c-4187-96c2-794b2107afeb admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.088769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba14dfe-8d2c-4187-96c2-794b2107afeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.089140 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba14dfe-8d2c-4187-96c2-794b2107afeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.090301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba14dfe-8d2c-4187-96c2-794b2107afeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.092731 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 741/1479] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:22 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:23.093652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c17ea6-de93-44fb-8789-9a95aa03d976 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.093897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba14dfe-8d2c-4187-96c2-794b2107afeb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.105442 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c17ea6-de93-44fb-8789-9a95aa03d976 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.122191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6a9f6-f4c6-4ec1-ad4e-8e47f3e57450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.122191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6a9f6-f4c6-4ec1-ad4e-8e47f3e57450 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.122191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6a9f6-f4c6-4ec1-ad4e-8e47f3e57450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.122191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6a9f6-f4c6-4ec1-ad4e-8e47f3e57450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.122755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6a9f6-f4c6-4ec1-ad4e-8e47f3e57450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.122755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6a9f6-f4c6-4ec1-ad4e-8e47f3e57450 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.132614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 742/1480] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:23.132614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6a9f6-f4c6-4ec1-ad4e-8e47f3e57450 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.134125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 739/1481] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:23 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:23.134531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9ce6a9f6-f4c6-4ec1-ad4e-8e47f3e57450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:23.135304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba14dfe-8d2c-4187-96c2-794b2107afeb admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.146843 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:23.149109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6a9f6-f4c6-4ec1-ad4e-8e47f3e57450 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.227771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba14dfe-8d2c-4187-96c2-794b2107afeb admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.233098 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.233756 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.234040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.234356 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] PATH_INFO: `/v3/projects/c12755f4a61d4619b9dd5fee3cc5e695` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.237850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12755f4a61d4619b9dd5fee3cc5e695)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.240031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.244460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a9e06-def2-4d7f-805f-dbd828965a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.245231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a9e06-def2-4d7f-805f-dbd828965a3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.245402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a9e06-def2-4d7f-805f-dbd828965a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.245675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a9e06-def2-4d7f-805f-dbd828965a3d admin admin] PATH_INFO: `/v3/projects/53a063277f4e44c2b7512c0337b0e125/users/82d36316f8ad44af9dc86008205670bd/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.295966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a9e06-def2-4d7f-805f-dbd828965a3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a063277f4e44c2b7512c0337b0e125, user_id=82d36316f8ad44af9dc86008205670bd, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.301828 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a9e06-def2-4d7f-805f-dbd828965a3d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.332068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 740/1482] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:23 2022] PUT /identity/v3/projects/53a063277f4e44c2b7512c0337b0e125/users/82d36316f8ad44af9dc86008205670bd/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:23.332891 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a9e06-def2-4d7f-805f-dbd828965a3d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.341822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12755f4a61d4619b9dd5fee3cc5e695', 'request_id': 'req-41ffefd7-21f1-47bd-b617-82a05fd609b7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:23.348325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12755f4a61d4619b9dd5fee3cc5e695', 'request_id': 'req-41ffefd7-21f1-47bd-b617-82a05fd609b7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:23.359682 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12755f4a61d4619b9dd5fee3cc5e695', 'request_id': 'req-41ffefd7-21f1-47bd-b617-82a05fd609b7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:23.377353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] The token cache is being invalidate because project c12755f4a61d4619b9dd5fee3cc5e695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:23.377738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:23.380903 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:23.382645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 743/1483] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:23 2022] DELETE /identity/v3/projects/c12755f4a61d4619b9dd5fee3cc5e695 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:23.383538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.385351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:23.442907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a9e06-def2-4d7f-805f-dbd828965a3d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.447980 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b7d0c-da3c-4c6b-8fe1-2412e8139295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.448701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b7d0c-da3c-4c6b-8fe1-2412e8139295 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.449080 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b7d0c-da3c-4c6b-8fe1-2412e8139295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.449991 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b7d0c-da3c-4c6b-8fe1-2412e8139295 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.450799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b7d0c-da3c-4c6b-8fe1-2412e8139295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.453454 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b7d0c-da3c-4c6b-8fe1-2412e8139295 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.455088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:23.459331 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffefd7-21f1-47bd-b617-82a05fd609b7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.468368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a42bd-d2e6-428f-a9cf-5e2f46963b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.469377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a42bd-d2e6-428f-a9cf-5e2f46963b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.470064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a42bd-d2e6-428f-a9cf-5e2f46963b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.470720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a42bd-d2e6-428f-a9cf-5e2f46963b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.472042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a42bd-d2e6-428f-a9cf-5e2f46963b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.474150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a42bd-d2e6-428f-a9cf-5e2f46963b77 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.480787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 741/1484] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:23 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:23.482408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866dab2-6943-4955-b1bb-9ca6880a3ee1 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.484086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866dab2-6943-4955-b1bb-9ca6880a3ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.484515 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866dab2-6943-4955-b1bb-9ca6880a3ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.489040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 744/1485] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:23.489886 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a42bd-d2e6-428f-a9cf-5e2f46963b77 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.491741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-249a42bd-d2e6-428f-a9cf-5e2f46963b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:23.507507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a42bd-d2e6-428f-a9cf-5e2f46963b77 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.512579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac64fdc-6d27-473b-9d31-6e9f000d89f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.512996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0866dab2-6943-4955-b1bb-9ca6880a3ee1 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:23.513574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac64fdc-6d27-473b-9d31-6e9f000d89f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.513574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac64fdc-6d27-473b-9d31-6e9f000d89f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.513761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac64fdc-6d27-473b-9d31-6e9f000d89f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.514539 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac64fdc-6d27-473b-9d31-6e9f000d89f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.515519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0866dab2-6943-4955-b1bb-9ca6880a3ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:23.516183 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac64fdc-6d27-473b-9d31-6e9f000d89f8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.526213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 742/1486] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:23.527907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866dab2-6943-4955-b1bb-9ca6880a3ee1 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.538991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 745/1487] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:23.538991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb75e1f-b8b1-4979-802e-eff7632b68cd None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.538991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb75e1f-b8b1-4979-802e-eff7632b68cd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.539334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb75e1f-b8b1-4979-802e-eff7632b68cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.547423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866dab2-6943-4955-b1bb-9ca6880a3ee1 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.562026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fdb75e1f-b8b1-4979-802e-eff7632b68cd None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:23.563250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fdb75e1f-b8b1-4979-802e-eff7632b68cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:23.564495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1219fe3-13a7-4f4d-84be-56b1e5b91643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.565504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1219fe3-13a7-4f4d-84be-56b1e5b91643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.565818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1219fe3-13a7-4f4d-84be-56b1e5b91643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.566268 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1219fe3-13a7-4f4d-84be-56b1e5b91643 admin admin] PATH_INFO: `/v3/projects/53a063277f4e44c2b7512c0337b0e125/users/82d36316f8ad44af9dc86008205670bd/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.574191 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1219fe3-13a7-4f4d-84be-56b1e5b91643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a063277f4e44c2b7512c0337b0e125, user_id=82d36316f8ad44af9dc86008205670bd, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.576123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 746/1488] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:23.576990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb75e1f-b8b1-4979-802e-eff7632b68cd None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.577288 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1219fe3-13a7-4f4d-84be-56b1e5b91643 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.588489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb75e1f-b8b1-4979-802e-eff7632b68cd None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.591400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 743/1489] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:23 2022] PUT /identity/v3/projects/53a063277f4e44c2b7512c0337b0e125/users/82d36316f8ad44af9dc86008205670bd/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:23.592349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1219fe3-13a7-4f4d-84be-56b1e5b91643 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.659167 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b481c-9764-42f9-8815-d3859297f101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.660063 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b481c-9764-42f9-8815-d3859297f101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.660175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b481c-9764-42f9-8815-d3859297f101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.660466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b481c-9764-42f9-8815-d3859297f101 admin admin] PATH_INFO: `/v3/projects/1d34f54ed05a480a909d47cc1139c2e8/users/951c854a2c334014bfe1c7f2d788292e/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.668852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1219fe3-13a7-4f4d-84be-56b1e5b91643 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.674713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3cf67-ec0c-46a3-8cab-88607f272328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.675330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3cf67-ec0c-46a3-8cab-88607f272328 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.675772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3cf67-ec0c-46a3-8cab-88607f272328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.676049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3cf67-ec0c-46a3-8cab-88607f272328 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.676793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3cf67-ec0c-46a3-8cab-88607f272328 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:23.677001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3cf67-ec0c-46a3-8cab-88607f272328 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.678638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3cf67-ec0c-46a3-8cab-88607f272328 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.679606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b481c-9764-42f9-8815-d3859297f101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d34f54ed05a480a909d47cc1139c2e8, user_id=951c854a2c334014bfe1c7f2d788292e, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.680460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:23.684421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b481c-9764-42f9-8815-d3859297f101 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.701263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 744/1490] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:42:23.701263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe93da-27dc-4529-987c-d364a6259926 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.701263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe93da-27dc-4529-987c-d364a6259926 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.701263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe93da-27dc-4529-987c-d364a6259926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.704022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 747/1491] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:23 2022] PUT /identity/v3/projects/1d34f54ed05a480a909d47cc1139c2e8/users/951c854a2c334014bfe1c7f2d788292e/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:23.705177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b481c-9764-42f9-8815-d3859297f101 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.734834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-febe93da-27dc-4529-987c-d364a6259926 None None] MFA Rules not processed for user `82d36316f8ad44af9dc86008205670bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:23.737316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-febe93da-27dc-4529-987c-d364a6259926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:23.785161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b481c-9764-42f9-8815-d3859297f101 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.792606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a4d77-718f-4ab7-95a2-f4accb914ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.793373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a4d77-718f-4ab7-95a2-f4accb914ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.793655 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a4d77-718f-4ab7-95a2-f4accb914ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.793918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a4d77-718f-4ab7-95a2-f4accb914ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.794730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a4d77-718f-4ab7-95a2-f4accb914ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.796787 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a4d77-718f-4ab7-95a2-f4accb914ea2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.822242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 745/1492] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:23.826554 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe93da-27dc-4529-987c-d364a6259926 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.831249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 748/1493] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:23 2022] POST /identity/v3/projects => generated 401 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:23.835402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a4d77-718f-4ab7-95a2-f4accb914ea2 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.839952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe93da-27dc-4529-987c-d364a6259926 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.899034 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:23.914344 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee209e3f-950e-41ac-9000-428664c0cf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.915067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee209e3f-950e-41ac-9000-428664c0cf48 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.915460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee209e3f-950e-41ac-9000-428664c0cf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.915860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee209e3f-950e-41ac-9000-428664c0cf48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.916568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee209e3f-950e-41ac-9000-428664c0cf48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.918347 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee209e3f-950e-41ac-9000-428664c0cf48 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.921370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a4d77-718f-4ab7-95a2-f4accb914ea2 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:23.926828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cb0d2-7078-4fa9-bfb8-296fbaf55b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:23.927455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cb0d2-7078-4fa9-bfb8-296fbaf55b20 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:23.927737 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cb0d2-7078-4fa9-bfb8-296fbaf55b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:23.928135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cb0d2-7078-4fa9-bfb8-296fbaf55b20 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:23.928641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cb0d2-7078-4fa9-bfb8-296fbaf55b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:23.930612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cb0d2-7078-4fa9-bfb8-296fbaf55b20 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:23.935261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 746/1494] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:23.935261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee209e3f-950e-41ac-9000-428664c0cf48 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:23.935261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ee209e3f-950e-41ac-9000-428664c0cf48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:23.968914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 749/1495] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:23 2022] POST /identity/v3/users => generated 301 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:42:23.984837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cb0d2-7078-4fa9-bfb8-296fbaf55b20 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:24.016202 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cb0d2-7078-4fa9-bfb8-296fbaf55b20 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:24.033459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a8a94-b3b4-4bc6-b78f-790fde8bbcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:24.034577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a8a94-b3b4-4bc6-b78f-790fde8bbcbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:24.034861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a8a94-b3b4-4bc6-b78f-790fde8bbcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:24.035134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a8a94-b3b4-4bc6-b78f-790fde8bbcbb admin admin] PATH_INFO: `/v3/projects/1d34f54ed05a480a909d47cc1139c2e8/users/951c854a2c334014bfe1c7f2d788292e/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:24.036460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee209e3f-950e-41ac-9000-428664c0cf48 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:24.041063 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a8a94-b3b4-4bc6-b78f-790fde8bbcbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d34f54ed05a480a909d47cc1139c2e8, user_id=951c854a2c334014bfe1c7f2d788292e, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:24.041535 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd5c5a-2f0a-4998-a68b-93faec0c00db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:24.042802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd5c5a-2f0a-4998-a68b-93faec0c00db service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:24.043163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd5c5a-2f0a-4998-a68b-93faec0c00db service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:24.043690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd5c5a-2f0a-4998-a68b-93faec0c00db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:24.043926 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a8a94-b3b4-4bc6-b78f-790fde8bbcbb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:24.083659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 750/1496] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:23 2022] PUT /identity/v3/projects/1d34f54ed05a480a909d47cc1139c2e8/users/951c854a2c334014bfe1c7f2d788292e/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:24.083905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:24.084637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a8a94-b3b4-4bc6-b78f-790fde8bbcbb admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:24.136883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd5c5a-2f0a-4998-a68b-93faec0c00db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:24.138410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd5c5a-2f0a-4998-a68b-93faec0c00db service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:24.167604 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 747/1497] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:24.169375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3e935-0a9d-4046-9bad-6e478747c8f4 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:24.169662 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3e935-0a9d-4046-9bad-6e478747c8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:24.170601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3e935-0a9d-4046-9bad-6e478747c8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:24.192928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a8a94-b3b4-4bc6-b78f-790fde8bbcbb admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:24.211849 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30992504-152d-44b5-9a22-2a1f6ed98314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:24.212603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30992504-152d-44b5-9a22-2a1f6ed98314 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:24.212751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30992504-152d-44b5-9a22-2a1f6ed98314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:24.213037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30992504-152d-44b5-9a22-2a1f6ed98314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:24.213849 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30992504-152d-44b5-9a22-2a1f6ed98314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:24.215853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30992504-152d-44b5-9a22-2a1f6ed98314 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:24.242209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 751/1498] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:24.247501 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30992504-152d-44b5-9a22-2a1f6ed98314 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:24.253634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f5f3e935-0a9d-4046-9bad-6e478747c8f4 None None] MFA Rules not processed for user `951c854a2c334014bfe1c7f2d788292e`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:24.255477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f5f3e935-0a9d-4046-9bad-6e478747c8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:24.282522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30992504-152d-44b5-9a22-2a1f6ed98314 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:24.294855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a699d-7140-4d96-ba11-796fbf9f35b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:24.295542 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a699d-7140-4d96-ba11-796fbf9f35b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:24.296736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a699d-7140-4d96-ba11-796fbf9f35b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:24.297170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a699d-7140-4d96-ba11-796fbf9f35b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:24.297815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a699d-7140-4d96-ba11-796fbf9f35b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:24.300115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a699d-7140-4d96-ba11-796fbf9f35b9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:24.312359 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 752/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:24.313686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a699d-7140-4d96-ba11-796fbf9f35b9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:24.317245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c03a699d-7140-4d96-ba11-796fbf9f35b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:24.335757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 748/1500] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:24.336839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3e935-0a9d-4046-9bad-6e478747c8f4 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:24.356566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3e935-0a9d-4046-9bad-6e478747c8f4 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:24.364685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7024a2-ad1d-4cfa-b8b0-c001d7e5859e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:24.365787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7024a2-ad1d-4cfa-b8b0-c001d7e5859e admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:24.366336 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7024a2-ad1d-4cfa-b8b0-c001d7e5859e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:24.366707 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7024a2-ad1d-4cfa-b8b0-c001d7e5859e admin admin] PATH_INFO: `/v3/projects/a08dade49581425d8a4a46d8776ae9f4/users/b2cb2d2f047b46d58d2a05ab0807240d/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:24.393903 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7024a2-ad1d-4cfa-b8b0-c001d7e5859e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a08dade49581425d8a4a46d8776ae9f4, user_id=b2cb2d2f047b46d58d2a05ab0807240d, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:24.393903 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7024a2-ad1d-4cfa-b8b0-c001d7e5859e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:24.399288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a699d-7140-4d96-ba11-796fbf9f35b9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:24.404124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f3c12-112c-4259-8ea6-d9b81ffe7e07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:24.404801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f3c12-112c-4259-8ea6-d9b81ffe7e07 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:24.405078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f3c12-112c-4259-8ea6-d9b81ffe7e07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:24.405595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f3c12-112c-4259-8ea6-d9b81ffe7e07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:24.409070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fa6f3c12-112c-4259-8ea6-d9b81ffe7e07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:24.416014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 749/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:24 2022] PUT /identity/v3/projects/a08dade49581425d8a4a46d8776ae9f4/users/b2cb2d2f047b46d58d2a05ab0807240d/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:24.417264 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7024a2-ad1d-4cfa-b8b0-c001d7e5859e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:24.497920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f3c12-112c-4259-8ea6-d9b81ffe7e07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:24.499583 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f3c12-112c-4259-8ea6-d9b81ffe7e07 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:24.504986 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:24.507918 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7024a2-ad1d-4cfa-b8b0-c001d7e5859e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:24.511559 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 753/1502] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:24.513043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad31794-e921-4f7d-92cf-645153c8c762 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:24.513427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad31794-e921-4f7d-92cf-645153c8c762 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:24.513693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad31794-e921-4f7d-92cf-645153c8c762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:24.515725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b954e-3b23-4561-baae-963d937f7b49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:24.516976 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b954e-3b23-4561-baae-963d937f7b49 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:24.517533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b954e-3b23-4561-baae-963d937f7b49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:24.518053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b954e-3b23-4561-baae-963d937f7b49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:24.554154 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4ad31794-e921-4f7d-92cf-645153c8c762 None None] MFA Rules not processed for user `b2cb2d2f047b46d58d2a05ab0807240d`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:24.556124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4ad31794-e921-4f7d-92cf-645153c8c762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:24.574244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:24.612806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b954e-3b23-4561-baae-963d937f7b49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:24.614454 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b954e-3b23-4561-baae-963d937f7b49 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:24.625932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 750/1503] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:24.662230 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 754/1504] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:24 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:24.666232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b954e-3b23-4561-baae-963d937f7b49 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:24.763297 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b954e-3b23-4561-baae-963d937f7b49 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:24.777666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5f28b-ce55-426f-bc17-70c8dccc316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:24.778444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5f28b-ce55-426f-bc17-70c8dccc316e admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:24.778822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5f28b-ce55-426f-bc17-70c8dccc316e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:24.779179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5f28b-ce55-426f-bc17-70c8dccc316e admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:24.779951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5f28b-ce55-426f-bc17-70c8dccc316e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:24.782313 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5f28b-ce55-426f-bc17-70c8dccc316e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:24.847347 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 751/1505] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:24 2022] POST /identity/v3/projects => generated 399 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:42:24.851094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad31794-e921-4f7d-92cf-645153c8c762 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:24.873249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad31794-e921-4f7d-92cf-645153c8c762 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:24.978140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50ca52-921a-485d-b459-b308bbe055d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:24.979201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50ca52-921a-485d-b459-b308bbe055d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:24.980127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50ca52-921a-485d-b459-b308bbe055d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:24.980490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50ca52-921a-485d-b459-b308bbe055d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:24.981263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50ca52-921a-485d-b459-b308bbe055d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:24.983757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50ca52-921a-485d-b459-b308bbe055d5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:25.009200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 755/1506] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:24 2022] POST /identity/v3/users => generated 306 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:25.013227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5f28b-ce55-426f-bc17-70c8dccc316e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:25.025269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5f28b-ce55-426f-bc17-70c8dccc316e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:25.030098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c1b71-d5a1-4c42-9b88-4e85f3d5eff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:25.030778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1b71-d5a1-4c42-9b88-4e85f3d5eff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:25.031057 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1b71-d5a1-4c42-9b88-4e85f3d5eff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:25.031322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1b71-d5a1-4c42-9b88-4e85f3d5eff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:25.031971 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c1b71-d5a1-4c42-9b88-4e85f3d5eff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:25.034188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c1b71-d5a1-4c42-9b88-4e85f3d5eff5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:25.045617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 752/1507] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:25.050328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50ca52-921a-485d-b459-b308bbe055d5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:25.061657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50ca52-921a-485d-b459-b308bbe055d5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:25.074340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba17da-195a-4aa7-8f83-a731c0f06fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:25.074876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c1b71-d5a1-4c42-9b88-4e85f3d5eff5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:25.075052 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba17da-195a-4aa7-8f83-a731c0f06fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:25.075328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba17da-195a-4aa7-8f83-a731c0f06fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:25.075737 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba17da-195a-4aa7-8f83-a731c0f06fb0 admin admin] PATH_INFO: `/v3/projects/e6d7c031d02649f8bf19fe11424dd2b3/users/7396ddcc12614b27a4aa26780e40b1a7/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:25.095814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba17da-195a-4aa7-8f83-a731c0f06fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d7c031d02649f8bf19fe11424dd2b3, user_id=7396ddcc12614b27a4aa26780e40b1a7, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:25.098690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba17da-195a-4aa7-8f83-a731c0f06fb0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:25.117439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 756/1508] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:25 2022] PUT /identity/v3/projects/e6d7c031d02649f8bf19fe11424dd2b3/users/7396ddcc12614b27a4aa26780e40b1a7/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:25.120624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba17da-195a-4aa7-8f83-a731c0f06fb0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:25.150498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c1b71-d5a1-4c42-9b88-4e85f3d5eff5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:25.190713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:25.198493 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba17da-195a-4aa7-8f83-a731c0f06fb0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:25.203715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1405ad9-f948-41a6-bfe2-60fc955318a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:25.204332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1405ad9-f948-41a6-bfe2-60fc955318a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:25.204643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1405ad9-f948-41a6-bfe2-60fc955318a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:25.204951 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1405ad9-f948-41a6-bfe2-60fc955318a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:25.205083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:25.205943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1405ad9-f948-41a6-bfe2-60fc955318a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:25.207665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1405ad9-f948-41a6-bfe2-60fc955318a0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:25.226223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9a2a3-7a1a-4e2a-b597-735ecbedd293 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '951c854a2c334014bfe1c7f2d788292e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d34f54ed05a480a909d47cc1139c2e8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:25.226223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9a2a3-7a1a-4e2a-b597-735ecbedd293 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:25.226223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9a2a3-7a1a-4e2a-b597-735ecbedd293 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:25.226611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9a2a3-7a1a-4e2a-b597-735ecbedd293 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:25.226611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9a2a3-7a1a-4e2a-b597-735ecbedd293 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:25.226734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9a2a3-7a1a-4e2a-b597-735ecbedd293 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:25.230270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 757/1509] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:42:25.231972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9a2a3-7a1a-4e2a-b597-735ecbedd293 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:25.232313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1405ad9-f948-41a6-bfe2-60fc955318a0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:25.257341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1405ad9-f948-41a6-bfe2-60fc955318a0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:25.262230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 753/1510] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:42:25.266621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02251ce1-0547-4aca-856d-63fe7309e37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:25.267533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02251ce1-0547-4aca-856d-63fe7309e37b admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:25.268049 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02251ce1-0547-4aca-856d-63fe7309e37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:25.268457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02251ce1-0547-4aca-856d-63fe7309e37b admin admin] PATH_INFO: `/v3/projects/e6d7c031d02649f8bf19fe11424dd2b3/users/7396ddcc12614b27a4aa26780e40b1a7/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:25.271744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9a2a3-7a1a-4e2a-b597-735ecbedd293 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:25.272735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02251ce1-0547-4aca-856d-63fe7309e37b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d7c031d02649f8bf19fe11424dd2b3, user_id=7396ddcc12614b27a4aa26780e40b1a7, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:25.275473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02251ce1-0547-4aca-856d-63fe7309e37b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:25.290213 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 758/1511] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:25 2022] PUT /identity/v3/projects/e6d7c031d02649f8bf19fe11424dd2b3/users/7396ddcc12614b27a4aa26780e40b1a7/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:25.300720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5836f-51c2-47dc-bb4d-986cac912a7a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:25.301021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5836f-51c2-47dc-bb4d-986cac912a7a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:25.301137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9a2a3-7a1a-4e2a-b597-735ecbedd293 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:25.305496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5836f-51c2-47dc-bb4d-986cac912a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:25.353328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d0c5836f-51c2-47dc-bb4d-986cac912a7a None None] MFA Rules not processed for user `7396ddcc12614b27a4aa26780e40b1a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:25.354587 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d0c5836f-51c2-47dc-bb4d-986cac912a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:25.428893 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ec96f-8cb2-41aa-8bfb-6cc5691ede89 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '951c854a2c334014bfe1c7f2d788292e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d34f54ed05a480a909d47cc1139c2e8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:25.429505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ec96f-8cb2-41aa-8bfb-6cc5691ede89 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:25.429783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ec96f-8cb2-41aa-8bfb-6cc5691ede89 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:25.430048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ec96f-8cb2-41aa-8bfb-6cc5691ede89 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:25.431343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ec96f-8cb2-41aa-8bfb-6cc5691ede89 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:25.434338 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ec96f-8cb2-41aa-8bfb-6cc5691ede89 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:25.471921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 754/1512] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:25 2022] POST /identity/v3/projects => generated 353 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:42:25.476139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ec96f-8cb2-41aa-8bfb-6cc5691ede89 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:25.482679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 759/1513] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:25.492690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5836f-51c2-47dc-bb4d-986cac912a7a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:25.581055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5836f-51c2-47dc-bb4d-986cac912a7a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:25.672898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b29b4d-738d-4f7e-b47a-c01abc84a5c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:25.674988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b29b4d-738d-4f7e-b47a-c01abc84a5c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:25.675459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b29b4d-738d-4f7e-b47a-c01abc84a5c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:25.677822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b29b4d-738d-4f7e-b47a-c01abc84a5c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:25.721272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:25.736901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ec96f-8cb2-41aa-8bfb-6cc5691ede89 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:25.742727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2143722-ce8b-4ef8-8c09-e660df122ce5 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '951c854a2c334014bfe1c7f2d788292e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d34f54ed05a480a909d47cc1139c2e8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:25.743431 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2143722-ce8b-4ef8-8c09-e660df122ce5 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:25.743782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2143722-ce8b-4ef8-8c09-e660df122ce5 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:25.744123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2143722-ce8b-4ef8-8c09-e660df122ce5 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:25.744936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2143722-ce8b-4ef8-8c09-e660df122ce5 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:25.745311 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2143722-ce8b-4ef8-8c09-e660df122ce5 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:25.747055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2143722-ce8b-4ef8-8c09-e660df122ce5 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:25.768010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 755/1514] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:42:25.809069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b29b4d-738d-4f7e-b47a-c01abc84a5c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:25.810944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b29b4d-738d-4f7e-b47a-c01abc84a5c1 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:25.834230 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 760/1515] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:42:25.848613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2143722-ce8b-4ef8-8c09-e660df122ce5 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:25.870057 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2143722-ce8b-4ef8-8c09-e660df122ce5 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:25.886297 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '951c854a2c334014bfe1c7f2d788292e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d34f54ed05a480a909d47cc1139c2e8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:25.887032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:25.887316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:25.887644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] PATH_INFO: `/v3/projects/b4f2ad814787491b802c426f84698583` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:25.888911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b4f2ad814787491b802c426f84698583)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:25.890821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:25.995220 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b29b4d-738d-4f7e-b47a-c01abc84a5c1 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:25.998145 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4f2ad814787491b802c426f84698583', 'request_id': 'req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:26.006769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b29b4d-738d-4f7e-b47a-c01abc84a5c1 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:26.008566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4f2ad814787491b802c426f84698583', 'request_id': 'req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:26.021500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4f2ad814787491b802c426f84698583', 'request_id': 'req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:26.027415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:26.042020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] The token cache is being invalidate because project b4f2ad814787491b802c426f84698583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:26.054999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:26.059086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:26.065970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 756/1516] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:25 2022] DELETE /identity/v3/projects/b4f2ad814787491b802c426f84698583 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:26.121709 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4bbba-772c-4570-9d49-02beb1bfb76c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:26.122514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4bbba-772c-4570-9d49-02beb1bfb76c service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:26.122801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4bbba-772c-4570-9d49-02beb1bfb76c service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:26.123078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4bbba-772c-4570-9d49-02beb1bfb76c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:26.124898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0ab4bbba-772c-4570-9d49-02beb1bfb76c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:26.205857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4bbba-772c-4570-9d49-02beb1bfb76c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:26.207378 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4bbba-772c-4570-9d49-02beb1bfb76c service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:26.220880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 761/1517] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:25 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:26.733251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:26.734759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:26.822005 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1ef49-a0e9-4eed-b6ea-cd97d0e15175 tempest-VolumeQuotaClassesTest-1801891352 tempest-VolumeQuotaClassesTest-1801891352-project-admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:26.827903 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:26.828704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:26.829182 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:26.830122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] PATH_INFO: `/v3/users/4c73b1acf31d4ef48368c9fc8c0890bc` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:26.831584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c73b1acf31d4ef48368c9fc8c0890bc)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:26.833646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:26.900372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c73b1acf31d4ef48368c9fc8c0890bc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:26.901136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c73b1acf31d4ef48368c9fc8c0890bc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:26.901911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c73b1acf31d4ef48368c9fc8c0890bc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:26.913294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c73b1acf31d4ef48368c9fc8c0890bc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:26.935600 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c73b1acf31d4ef48368c9fc8c0890bc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:26.960468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 757/1518] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:26 2022] DELETE /identity/v3/users/4c73b1acf31d4ef48368c9fc8c0890bc => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:26.965543 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec583e-8bc7-4a64-aa0e-bb2fffb0f998 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:26.965837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec583e-8bc7-4a64-aa0e-bb2fffb0f998 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:26.966127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec583e-8bc7-4a64-aa0e-bb2fffb0f998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:27.003340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c7ec583e-8bc7-4a64-aa0e-bb2fffb0f998 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:27.004760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c7ec583e-8bc7-4a64-aa0e-bb2fffb0f998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:27.104616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 762/1519] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:27.117262 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:27.119540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:27.210285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f114d3-2aac-425f-ab33-822e43b74163 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:27.213928 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109e98c-b8ae-4e5a-9e0f-f449ba2b3a87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:27.214592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109e98c-b8ae-4e5a-9e0f-f449ba2b3a87 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:27.215252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109e98c-b8ae-4e5a-9e0f-f449ba2b3a87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:27.215534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109e98c-b8ae-4e5a-9e0f-f449ba2b3a87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:27.218976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5874d13-eed9-4954-862b-752a583fe347 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:27.219267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5874d13-eed9-4954-862b-752a583fe347 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:27.219537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5874d13-eed9-4954-862b-752a583fe347 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:27.221293 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 763/1520] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:42:27.228667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109e98c-b8ae-4e5a-9e0f-f449ba2b3a87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:27.228986 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e587-1143-4d3f-b5cb-72c16ff3d03b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:27.229373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e587-1143-4d3f-b5cb-72c16ff3d03b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:27.229718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e587-1143-4d3f-b5cb-72c16ff3d03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:27.230725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109e98c-b8ae-4e5a-9e0f-f449ba2b3a87 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:27.245111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 758/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:27.258470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fa96e587-1143-4d3f-b5cb-72c16ff3d03b None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:27.259686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fa96e587-1143-4d3f-b5cb-72c16ff3d03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:27.304196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:27.385790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 764/1522] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:42:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:27.399620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109e98c-b8ae-4e5a-9e0f-f449ba2b3a87 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:27.401111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5109e98c-b8ae-4e5a-9e0f-f449ba2b3a87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:27.419190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109e98c-b8ae-4e5a-9e0f-f449ba2b3a87 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:27.428306 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a548e-1221-43ff-8fc2-3009d980b093 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:27.428859 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a548e-1221-43ff-8fc2-3009d980b093 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:27.429161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a548e-1221-43ff-8fc2-3009d980b093 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:27.429411 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a548e-1221-43ff-8fc2-3009d980b093 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:27.442327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a548e-1221-43ff-8fc2-3009d980b093 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:27.443933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a548e-1221-43ff-8fc2-3009d980b093 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:27.455919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 759/1523] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:27.683339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96e587-1143-4d3f-b5cb-72c16ff3d03b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:27.684884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fa96e587-1143-4d3f-b5cb-72c16ff3d03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:27.701752 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96e587-1143-4d3f-b5cb-72c16ff3d03b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:27.707175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:27.707828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:27.708262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:27.708813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] PATH_INFO: `/v3/projects/21a9da389eb640a89e9157992ef607ab` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:27.710996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a9da389eb640a89e9157992ef607ab)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:27.712962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:27.783787 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a9da389eb640a89e9157992ef607ab', 'request_id': 'req-3a39ff1f-e649-43d8-87d6-be0cc70b750b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:27.792325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a9da389eb640a89e9157992ef607ab', 'request_id': 'req-3a39ff1f-e649-43d8-87d6-be0cc70b750b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:27.804747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a9da389eb640a89e9157992ef607ab', 'request_id': 'req-3a39ff1f-e649-43d8-87d6-be0cc70b750b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:27.818716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] The token cache is being invalidate because project 21a9da389eb640a89e9157992ef607ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:27.819014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:27.819690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:27.830642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 765/1524] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:27 2022] DELETE /identity/v3/projects/21a9da389eb640a89e9157992ef607ab => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:27.835903 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a548e-1221-43ff-8fc2-3009d980b093 service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:27.850535 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:27.859321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a548e-1221-43ff-8fc2-3009d980b093 service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:27.935159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40d7a6-b333-482b-970c-284bab5a0e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:27.936241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40d7a6-b333-482b-970c-284bab5a0e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:27.936692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40d7a6-b333-482b-970c-284bab5a0e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:27.937254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40d7a6-b333-482b-970c-284bab5a0e16 admin admin] PATH_INFO: `/v3/users/951c854a2c334014bfe1c7f2d788292e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:27.938584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40d7a6-b333-482b-970c-284bab5a0e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=951c854a2c334014bfe1c7f2d788292e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:27.942136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40d7a6-b333-482b-970c-284bab5a0e16 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:27.996954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d40d7a6-b333-482b-970c-284bab5a0e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '951c854a2c334014bfe1c7f2d788292e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:27.997787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d40d7a6-b333-482b-970c-284bab5a0e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '951c854a2c334014bfe1c7f2d788292e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:27.998646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d40d7a6-b333-482b-970c-284bab5a0e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '951c854a2c334014bfe1c7f2d788292e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:28.006561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d40d7a6-b333-482b-970c-284bab5a0e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '951c854a2c334014bfe1c7f2d788292e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:28.023116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d40d7a6-b333-482b-970c-284bab5a0e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '951c854a2c334014bfe1c7f2d788292e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:28.040967 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 760/1525] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:27 2022] DELETE /identity/v3/users/951c854a2c334014bfe1c7f2d788292e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:28.473485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:28.475161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:28.491393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39ff1f-e649-43d8-87d6-be0cc70b750b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:28.550432 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:28.551474 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:28.552050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:28.552345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] PATH_INFO: `/v3/projects/1d34f54ed05a480a909d47cc1139c2e8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:28.553470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d34f54ed05a480a909d47cc1139c2e8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:28.555394 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:28.626486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:28.636277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d34f54ed05a480a909d47cc1139c2e8', 'request_id': 'req-3fcbdf76-d349-4808-b996-5dd4b39db288'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:28.649166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d34f54ed05a480a909d47cc1139c2e8', 'request_id': 'req-3fcbdf76-d349-4808-b996-5dd4b39db288'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:28.662192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d34f54ed05a480a909d47cc1139c2e8', 'request_id': 'req-3fcbdf76-d349-4808-b996-5dd4b39db288'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:28.676538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] The token cache is being invalidate because project 1d34f54ed05a480a909d47cc1139c2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:28.676883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:28.677636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3fcbdf76-d349-4808-b996-5dd4b39db288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:28.679895 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 766/1526] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:28 2022] DELETE /identity/v3/projects/1d34f54ed05a480a909d47cc1139c2e8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:28.685335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42c763-96a0-45dd-8925-c83d39605dfe None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:28.685746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42c763-96a0-45dd-8925-c83d39605dfe None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:28.686045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42c763-96a0-45dd-8925-c83d39605dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:28.713196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bb42c763-96a0-45dd-8925-c83d39605dfe None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:28.714634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bb42c763-96a0-45dd-8925-c83d39605dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:28.795100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc4e6a-bf46-4962-8b5a-12f2b194e5e2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:28.795735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc4e6a-bf46-4962-8b5a-12f2b194e5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:28.796128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc4e6a-bf46-4962-8b5a-12f2b194e5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:28.802066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 761/1527] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:28.818600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5ebc4e6a-bf46-4962-8b5a-12f2b194e5e2 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:28.819913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5ebc4e6a-bf46-4962-8b5a-12f2b194e5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:28.829861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 767/1528] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:28.834059 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42c763-96a0-45dd-8925-c83d39605dfe None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:28.845763 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42c763-96a0-45dd-8925-c83d39605dfe None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:28.853852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0a74e-19e4-4c8e-b22c-ba7b1da9b9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:28.854503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0a74e-19e4-4c8e-b22c-ba7b1da9b9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:28.854780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0a74e-19e4-4c8e-b22c-ba7b1da9b9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:28.855048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0a74e-19e4-4c8e-b22c-ba7b1da9b9d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:28.855873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0a74e-19e4-4c8e-b22c-ba7b1da9b9d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:28.856178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0a74e-19e4-4c8e-b22c-ba7b1da9b9d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:28.857763 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0a74e-19e4-4c8e-b22c-ba7b1da9b9d7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:28.884152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 762/1529] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:42:28.888545 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc4e6a-bf46-4962-8b5a-12f2b194e5e2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:28.899462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc4e6a-bf46-4962-8b5a-12f2b194e5e2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:28.904703 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751ca7b-b190-41ff-9259-a18da4ea9615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:28.905373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751ca7b-b190-41ff-9259-a18da4ea9615 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:28.905562 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751ca7b-b190-41ff-9259-a18da4ea9615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:28.905858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751ca7b-b190-41ff-9259-a18da4ea9615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:28.906518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751ca7b-b190-41ff-9259-a18da4ea9615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:28.909730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751ca7b-b190-41ff-9259-a18da4ea9615 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:28.954688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 768/1530] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:28 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:28.956688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0a74e-19e4-4c8e-b22c-ba7b1da9b9d7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:28.972802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0a74e-19e4-4c8e-b22c-ba7b1da9b9d7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:28.978449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:29.058099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ad80b-dcac-4559-a457-1a542f9c8e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:29.058797 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ad80b-dcac-4559-a457-1a542f9c8e47 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:29.059146 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ad80b-dcac-4559-a457-1a542f9c8e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:29.059485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ad80b-dcac-4559-a457-1a542f9c8e47 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:29.060190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ad80b-dcac-4559-a457-1a542f9c8e47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:29.062524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ad80b-dcac-4559-a457-1a542f9c8e47 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:29.105704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 763/1531] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:28 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:29.105917 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751ca7b-b190-41ff-9259-a18da4ea9615 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:29.121864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751ca7b-b190-41ff-9259-a18da4ea9615 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:29.136857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b841e-3ba6-4492-8b04-811a2fdf5710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:29.137512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b841e-3ba6-4492-8b04-811a2fdf5710 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:29.137796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b841e-3ba6-4492-8b04-811a2fdf5710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:29.138068 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b841e-3ba6-4492-8b04-811a2fdf5710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:29.138758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b841e-3ba6-4492-8b04-811a2fdf5710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:29.140505 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b841e-3ba6-4492-8b04-811a2fdf5710 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:29.191569 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 769/1532] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:29.192844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ad80b-dcac-4559-a457-1a542f9c8e47 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:29.230231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ad80b-dcac-4559-a457-1a542f9c8e47 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:29.244735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22ef3c-0855-4518-a091-cc8abd6d1568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:29.245364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22ef3c-0855-4518-a091-cc8abd6d1568 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:29.245642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22ef3c-0855-4518-a091-cc8abd6d1568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:29.245910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22ef3c-0855-4518-a091-cc8abd6d1568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:29.246589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22ef3c-0855-4518-a091-cc8abd6d1568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:29.254489 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22ef3c-0855-4518-a091-cc8abd6d1568 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:29.292125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 764/1533] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:29.296719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b841e-3ba6-4492-8b04-811a2fdf5710 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:29.317261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b841e-3ba6-4492-8b04-811a2fdf5710 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:29.355148 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340c0ea-8d10-4765-848e-b2e520b923ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:29.357233 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340c0ea-8d10-4765-848e-b2e520b923ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:29.357755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340c0ea-8d10-4765-848e-b2e520b923ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:29.361199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340c0ea-8d10-4765-848e-b2e520b923ec admin admin] PATH_INFO: `/v3/projects/3c91e8ff0bfc4f78a3165c3583c5085a/users/8596a908ae40431c8fa25c11b6ac9c7b/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:29.404736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340c0ea-8d10-4765-848e-b2e520b923ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c91e8ff0bfc4f78a3165c3583c5085a, user_id=8596a908ae40431c8fa25c11b6ac9c7b, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:29.410519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340c0ea-8d10-4765-848e-b2e520b923ec admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:29.437206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 770/1534] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:29 2022] PUT /identity/v3/projects/3c91e8ff0bfc4f78a3165c3583c5085a/users/8596a908ae40431c8fa25c11b6ac9c7b/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:29.451091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0301f-378b-4a22-87da-48646c4c8fdf None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:29.451373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0301f-378b-4a22-87da-48646c4c8fdf None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:29.451644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0301f-378b-4a22-87da-48646c4c8fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:29.545045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-fde0301f-378b-4a22-87da-48646c4c8fdf None None] MFA Rules not processed for user `8596a908ae40431c8fa25c11b6ac9c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:29.546284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fde0301f-378b-4a22-87da-48646c4c8fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:29.715126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 765/1535] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:29.720615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340c0ea-8d10-4765-848e-b2e520b923ec admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:29.805536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0301f-378b-4a22-87da-48646c4c8fdf None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:29.807197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fde0301f-378b-4a22-87da-48646c4c8fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:29.885205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340c0ea-8d10-4765-848e-b2e520b923ec admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:29.892615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f412c18-ebeb-4945-8a11-8c9ac922755a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:29.893626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f412c18-ebeb-4945-8a11-8c9ac922755a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:29.894054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f412c18-ebeb-4945-8a11-8c9ac922755a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:29.894672 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f412c18-ebeb-4945-8a11-8c9ac922755a admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:29.895553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f412c18-ebeb-4945-8a11-8c9ac922755a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:29.897901 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f412c18-ebeb-4945-8a11-8c9ac922755a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:29.915543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0301f-378b-4a22-87da-48646c4c8fdf None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:29.928430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91809e-e09b-4fa5-9379-528c62597b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:29.929926 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91809e-e09b-4fa5-9379-528c62597b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:29.930405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91809e-e09b-4fa5-9379-528c62597b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:29.931055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91809e-e09b-4fa5-9379-528c62597b64 admin admin] PATH_INFO: `/v3/users/de8b7fba46ee439fad655e504feb12fc` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:29.933175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91809e-e09b-4fa5-9379-528c62597b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de8b7fba46ee439fad655e504feb12fc)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:29.936341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91809e-e09b-4fa5-9379-528c62597b64 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:29.957407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 771/1536] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:29 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:29.958394 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f412c18-ebeb-4945-8a11-8c9ac922755a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:29.986953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:30.052766 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf91809e-e09b-4fa5-9379-528c62597b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de8b7fba46ee439fad655e504feb12fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:30.053744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf91809e-e09b-4fa5-9379-528c62597b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de8b7fba46ee439fad655e504feb12fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:30.054417 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf91809e-e09b-4fa5-9379-528c62597b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de8b7fba46ee439fad655e504feb12fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:30.059723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:30.065465 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf91809e-e09b-4fa5-9379-528c62597b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de8b7fba46ee439fad655e504feb12fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:30.078395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f412c18-ebeb-4945-8a11-8c9ac922755a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.082540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf91809e-e09b-4fa5-9379-528c62597b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de8b7fba46ee439fad655e504feb12fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:30.086070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4faf8b-784c-4bf1-9271-e344a2fdaa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.086725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4faf8b-784c-4bf1-9271-e344a2fdaa7f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.087061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4faf8b-784c-4bf1-9271-e344a2fdaa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.087260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4faf8b-784c-4bf1-9271-e344a2fdaa7f admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.087890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4faf8b-784c-4bf1-9271-e344a2fdaa7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.090294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4faf8b-784c-4bf1-9271-e344a2fdaa7f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.113093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 766/1537] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:29 2022] DELETE /identity/v3/users/de8b7fba46ee439fad655e504feb12fc => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:30.115289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f41e8-7c4d-4945-b89e-5176470b09ff None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.115803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f41e8-7c4d-4945-b89e-5176470b09ff None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.116012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f41e8-7c4d-4945-b89e-5176470b09ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.119983 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 772/1538] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:29 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:30.124799 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4faf8b-784c-4bf1-9271-e344a2fdaa7f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.131790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3b4faf8b-784c-4bf1-9271-e344a2fdaa7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:30.144649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4faf8b-784c-4bf1-9271-e344a2fdaa7f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.152074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-867f41e8-7c4d-4945-b89e-5176470b09ff None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:30.153367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-867f41e8-7c4d-4945-b89e-5176470b09ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:30.214541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ac3e-c34e-4500-ad11-7183b0e86f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.215195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ac3e-c34e-4500-ad11-7183b0e86f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.215503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ac3e-c34e-4500-ad11-7183b0e86f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.215781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367ac3e-c34e-4500-ad11-7183b0e86f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.216636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367ac3e-c34e-4500-ad11-7183b0e86f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.218540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367ac3e-c34e-4500-ad11-7183b0e86f1d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.221147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 767/1539] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:30.231194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f41e8-7c4d-4945-b89e-5176470b09ff None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.232613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-867f41e8-7c4d-4945-b89e-5176470b09ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:30.240278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 773/1540] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:30.241446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ac3e-c34e-4500-ad11-7183b0e86f1d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.254035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367ac3e-c34e-4500-ad11-7183b0e86f1d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.258594 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121d25e-9a2c-4fd4-85ec-72964d6a0333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.259341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121d25e-9a2c-4fd4-85ec-72964d6a0333 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.259706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121d25e-9a2c-4fd4-85ec-72964d6a0333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.260042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121d25e-9a2c-4fd4-85ec-72964d6a0333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.260751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121d25e-9a2c-4fd4-85ec-72964d6a0333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.262465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121d25e-9a2c-4fd4-85ec-72964d6a0333 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.275651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 774/1541] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:30.281230 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121d25e-9a2c-4fd4-85ec-72964d6a0333 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.292927 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121d25e-9a2c-4fd4-85ec-72964d6a0333 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.297609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00db12-1a75-4d3c-9c3f-53cdb5027205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.298967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00db12-1a75-4d3c-9c3f-53cdb5027205 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.299379 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00db12-1a75-4d3c-9c3f-53cdb5027205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.299713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00db12-1a75-4d3c-9c3f-53cdb5027205 admin admin] PATH_INFO: `/v3/projects/ebfbb5b488f54d75ac0cbb0965ad61a3/users/30ccb34c05fc4faaa63ec1249ed031ec/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.301323 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f41e8-7c4d-4945-b89e-5176470b09ff None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.304994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a29c9-f682-4611-80a4-1bd752f7c2f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.305602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a29c9-f682-4611-80a4-1bd752f7c2f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.305883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a29c9-f682-4611-80a4-1bd752f7c2f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.306154 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a29c9-f682-4611-80a4-1bd752f7c2f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.315779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00db12-1a75-4d3c-9c3f-53cdb5027205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebfbb5b488f54d75ac0cbb0965ad61a3, user_id=30ccb34c05fc4faaa63ec1249ed031ec, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.316052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a29c9-f682-4611-80a4-1bd752f7c2f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.317518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a29c9-f682-4611-80a4-1bd752f7c2f5 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.318458 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00db12-1a75-4d3c-9c3f-53cdb5027205 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.327663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 768/1542] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:30.333348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 775/1543] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:30 2022] PUT /identity/v3/projects/ebfbb5b488f54d75ac0cbb0965ad61a3/users/30ccb34c05fc4faaa63ec1249ed031ec/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:30.343518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd72efe-d6d6-4c34-8e36-7182a293eb2f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.343804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd72efe-d6d6-4c34-8e36-7182a293eb2f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.344079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd72efe-d6d6-4c34-8e36-7182a293eb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.373710 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-cbd72efe-d6d6-4c34-8e36-7182a293eb2f None None] MFA Rules not processed for user `30ccb34c05fc4faaa63ec1249ed031ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:30.374989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cbd72efe-d6d6-4c34-8e36-7182a293eb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:30.409955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:30.451694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 769/1544] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:30.456359 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00db12-1a75-4d3c-9c3f-53cdb5027205 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.525436 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00db12-1a75-4d3c-9c3f-53cdb5027205 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.530358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af6a6c-842a-46e1-83f5-36937f9af987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.531134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6a6c-842a-46e1-83f5-36937f9af987 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.531415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6a6c-842a-46e1-83f5-36937f9af987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.531714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af6a6c-842a-46e1-83f5-36937f9af987 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.532370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af6a6c-842a-46e1-83f5-36937f9af987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.535234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af6a6c-842a-46e1-83f5-36937f9af987 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.565940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 776/1545] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:30 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:30.567294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd72efe-d6d6-4c34-8e36-7182a293eb2f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.577859 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd72efe-d6d6-4c34-8e36-7182a293eb2f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.640886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc678e0c-6fbe-4483-a4e8-cbcd8e9adfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.641730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc678e0c-6fbe-4483-a4e8-cbcd8e9adfb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.641923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc678e0c-6fbe-4483-a4e8-cbcd8e9adfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.642114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc678e0c-6fbe-4483-a4e8-cbcd8e9adfb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.642779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc678e0c-6fbe-4483-a4e8-cbcd8e9adfb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.644714 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc678e0c-6fbe-4483-a4e8-cbcd8e9adfb5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.669254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 770/1546] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:30 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:30.672798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af6a6c-842a-46e1-83f5-36937f9af987 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.685045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af6a6c-842a-46e1-83f5-36937f9af987 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.690496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa509a-76be-4570-843f-bc50f674138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.691104 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa509a-76be-4570-843f-bc50f674138d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.691380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa509a-76be-4570-843f-bc50f674138d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.691653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa509a-76be-4570-843f-bc50f674138d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.692287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa509a-76be-4570-843f-bc50f674138d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.693867 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa509a-76be-4570-843f-bc50f674138d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.706499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 777/1547] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:30.706702 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc678e0c-6fbe-4483-a4e8-cbcd8e9adfb5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.708455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-dc678e0c-6fbe-4483-a4e8-cbcd8e9adfb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:30.710452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa509a-76be-4570-843f-bc50f674138d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.723668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa509a-76be-4570-843f-bc50f674138d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.725416 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc678e0c-6fbe-4483-a4e8-cbcd8e9adfb5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.728538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656d585-8a23-4809-9593-a05ae7cfec6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.729202 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656d585-8a23-4809-9593-a05ae7cfec6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.729483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656d585-8a23-4809-9593-a05ae7cfec6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.729863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656d585-8a23-4809-9593-a05ae7cfec6b admin admin] PATH_INFO: `/v3/projects/c4b72c177620487f803307ca7ff4f0c1/users/f554685b8ce54c5495ace9e470231c30/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.731625 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.734672 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.734951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.735216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] PATH_INFO: `/v3/projects/d96389473d9f4fcab128adc205986054` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.736570 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d96389473d9f4fcab128adc205986054)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.738912 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.743550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:30.746910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656d585-8a23-4809-9593-a05ae7cfec6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b72c177620487f803307ca7ff4f0c1, user_id=f554685b8ce54c5495ace9e470231c30, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.750158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656d585-8a23-4809-9593-a05ae7cfec6b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.765929 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 778/1548] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:30 2022] PUT /identity/v3/projects/c4b72c177620487f803307ca7ff4f0c1/users/f554685b8ce54c5495ace9e470231c30/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:30.770901 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656d585-8a23-4809-9593-a05ae7cfec6b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.792873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd96389473d9f4fcab128adc205986054', 'request_id': 'req-08aefee1-0d65-4e93-8499-4eb7086f99b5'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:30.802640 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd96389473d9f4fcab128adc205986054', 'request_id': 'req-08aefee1-0d65-4e93-8499-4eb7086f99b5'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:30.815001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd96389473d9f4fcab128adc205986054', 'request_id': 'req-08aefee1-0d65-4e93-8499-4eb7086f99b5'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:30.836067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] The token cache is being invalidate because project d96389473d9f4fcab128adc205986054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:30.836446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:30.837380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:30.841945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 771/1549] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:30 2022] DELETE /identity/v3/projects/d96389473d9f4fcab128adc205986054 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:30.843106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.844492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:30.866005 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656d585-8a23-4809-9593-a05ae7cfec6b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.909880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:30.945300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aefee1-0d65-4e93-8499-4eb7086f99b5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.949462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e474013-4ad7-46fa-8fa6-d58836b84101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.950079 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e474013-4ad7-46fa-8fa6-d58836b84101 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.950361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.950501 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e474013-4ad7-46fa-8fa6-d58836b84101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.950640 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e474013-4ad7-46fa-8fa6-d58836b84101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.951153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.951439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e474013-4ad7-46fa-8fa6-d58836b84101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.952027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.952311 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] PATH_INFO: `/v3/users/82d36316f8ad44af9dc86008205670bd` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:30.953284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e474013-4ad7-46fa-8fa6-d58836b84101 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.953490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82d36316f8ad44af9dc86008205670bd)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:30.956274 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:30.966469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 779/1550] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:42:30.970144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e474013-4ad7-46fa-8fa6-d58836b84101 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:30.973217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7e474013-4ad7-46fa-8fa6-d58836b84101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:30.990841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e474013-4ad7-46fa-8fa6-d58836b84101 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:30.996794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc148a4-8fff-4c63-bde2-fd5c38344766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:30.998028 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc148a4-8fff-4c63-bde2-fd5c38344766 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:30.998526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc148a4-8fff-4c63-bde2-fd5c38344766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:30.998630 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc148a4-8fff-4c63-bde2-fd5c38344766 admin admin] PATH_INFO: `/v3/projects/c4b72c177620487f803307ca7ff4f0c1/users/f554685b8ce54c5495ace9e470231c30/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.001130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc148a4-8fff-4c63-bde2-fd5c38344766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b72c177620487f803307ca7ff4f0c1, user_id=f554685b8ce54c5495ace9e470231c30, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:31.004527 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc148a4-8fff-4c63-bde2-fd5c38344766 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:31.022112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 780/1551] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:30 2022] PUT /identity/v3/projects/c4b72c177620487f803307ca7ff4f0c1/users/f554685b8ce54c5495ace9e470231c30/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:31.026974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82d36316f8ad44af9dc86008205670bd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:31.027732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82d36316f8ad44af9dc86008205670bd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:31.028504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82d36316f8ad44af9dc86008205670bd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:31.032184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb09c6-65bf-4c66-9a6a-1d92dbadb1e0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:31.032505 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb09c6-65bf-4c66-9a6a-1d92dbadb1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:31.032946 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb09c6-65bf-4c66-9a6a-1d92dbadb1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.036714 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82d36316f8ad44af9dc86008205670bd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:31.050498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82d36316f8ad44af9dc86008205670bd'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:31.068567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 772/1552] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:30 2022] DELETE /identity/v3/users/82d36316f8ad44af9dc86008205670bd => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:31.071516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-30bb09c6-65bf-4c66-9a6a-1d92dbadb1e0 None None] MFA Rules not processed for user `f554685b8ce54c5495ace9e470231c30`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:31.071516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-30bb09c6-65bf-4c66-9a6a-1d92dbadb1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:31.150807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 781/1553] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:31.167169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:31.169368 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:31.233242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a6c7f-a49f-40aa-a843-61aa26262b56 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:31.237856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb41d2-adb0-40a3-9e0b-e96e7e70fbb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:31.238554 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb41d2-adb0-40a3-9e0b-e96e7e70fbb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:31.238834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb41d2-adb0-40a3-9e0b-e96e7e70fbb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:31.239101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb41d2-adb0-40a3-9e0b-e96e7e70fbb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.248706 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb41d2-adb0-40a3-9e0b-e96e7e70fbb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:31.250133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb41d2-adb0-40a3-9e0b-e96e7e70fbb0 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:31.260870 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 773/1554] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:31.407859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb09c6-65bf-4c66-9a6a-1d92dbadb1e0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:31.409883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-30bb09c6-65bf-4c66-9a6a-1d92dbadb1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:31.425192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:31.487989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb09c6-65bf-4c66-9a6a-1d92dbadb1e0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:31.496081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:31.496757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:31.497030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:31.497295 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] PATH_INFO: `/v3/projects/53a063277f4e44c2b7512c0337b0e125` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.498456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a063277f4e44c2b7512c0337b0e125)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:31.500336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:31.550084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a063277f4e44c2b7512c0337b0e125', 'request_id': 'req-86f00e8a-b1e5-4326-b4d3-c89b349376b6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:31.556501 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a063277f4e44c2b7512c0337b0e125', 'request_id': 'req-86f00e8a-b1e5-4326-b4d3-c89b349376b6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:31.564431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a063277f4e44c2b7512c0337b0e125', 'request_id': 'req-86f00e8a-b1e5-4326-b4d3-c89b349376b6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:31.575966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] The token cache is being invalidate because project 53a063277f4e44c2b7512c0337b0e125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:31.576251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:31.576884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f00e8a-b1e5-4326-b4d3-c89b349376b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:31.578410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 782/1555] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:31 2022] DELETE /identity/v3/projects/53a063277f4e44c2b7512c0337b0e125 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:31.583460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d53c6d-d8f8-4439-9f44-08a36a329d64 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:31.583773 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d53c6d-d8f8-4439-9f44-08a36a329d64 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:31.584043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d53c6d-d8f8-4439-9f44-08a36a329d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.603919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-12d53c6d-d8f8-4439-9f44-08a36a329d64 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:31.605123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-12d53c6d-d8f8-4439-9f44-08a36a329d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:31.682210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 774/1556] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:31.686780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637b722-c49e-4cad-b5b5-3cbdbacbf0c9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:31.687135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637b722-c49e-4cad-b5b5-3cbdbacbf0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:31.687341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637b722-c49e-4cad-b5b5-3cbdbacbf0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.723171 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-2637b722-c49e-4cad-b5b5-3cbdbacbf0c9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:31.724552 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2637b722-c49e-4cad-b5b5-3cbdbacbf0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:31.733879 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 783/1557] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:31.738721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d53c6d-d8f8-4439-9f44-08a36a329d64 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:31.744765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:31.748676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d53c6d-d8f8-4439-9f44-08a36a329d64 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:31.756640 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f223f9-c670-4219-a8b9-6641948a9e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:31.757480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f223f9-c670-4219-a8b9-6641948a9e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:31.757889 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f223f9-c670-4219-a8b9-6641948a9e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:31.758246 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f223f9-c670-4219-a8b9-6641948a9e22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.759224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f223f9-c670-4219-a8b9-6641948a9e22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:31.759528 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f223f9-c670-4219-a8b9-6641948a9e22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:31.762736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f223f9-c670-4219-a8b9-6641948a9e22 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:31.775947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 775/1558] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:42:31.780025 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637b722-c49e-4cad-b5b5-3cbdbacbf0c9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:31.790828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637b722-c49e-4cad-b5b5-3cbdbacbf0c9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:31.795610 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c96cde-a782-40a0-898d-9f5731826750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:31.796223 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c96cde-a782-40a0-898d-9f5731826750 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:31.796491 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c96cde-a782-40a0-898d-9f5731826750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:31.796760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c96cde-a782-40a0-898d-9f5731826750 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.797391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c96cde-a782-40a0-898d-9f5731826750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:31.799899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c96cde-a782-40a0-898d-9f5731826750 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:31.829544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 784/1559] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:31 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:31.833618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f223f9-c670-4219-a8b9-6641948a9e22 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:31.846214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f223f9-c670-4219-a8b9-6641948a9e22 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:31.907086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da8546-b60b-4a85-8431-7970b54a5a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:31.908328 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da8546-b60b-4a85-8431-7970b54a5a04 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:31.908894 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da8546-b60b-4a85-8431-7970b54a5a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:31.909172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da8546-b60b-4a85-8431-7970b54a5a04 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.909742 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c96cde-a782-40a0-898d-9f5731826750 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:31.909888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da8546-b60b-4a85-8431-7970b54a5a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:31.911756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da8546-b60b-4a85-8431-7970b54a5a04 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:31.913507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d9c96cde-a782-40a0-898d-9f5731826750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:31.933996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 776/1560] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:31 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:31.938063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da8546-b60b-4a85-8431-7970b54a5a04 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:31.949282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da8546-b60b-4a85-8431-7970b54a5a04 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:31.953909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffcbab-5fcd-49c9-a71b-c13fcf95dd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:31.954636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffcbab-5fcd-49c9-a71b-c13fcf95dd75 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:31.955126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffcbab-5fcd-49c9-a71b-c13fcf95dd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:31.955477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffcbab-5fcd-49c9-a71b-c13fcf95dd75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.956110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffcbab-5fcd-49c9-a71b-c13fcf95dd75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:31.957681 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffcbab-5fcd-49c9-a71b-c13fcf95dd75 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:31.968202 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 777/1561] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:42:31.972605 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffcbab-5fcd-49c9-a71b-c13fcf95dd75 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:31.979143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c96cde-a782-40a0-898d-9f5731826750 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:31.982137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffcbab-5fcd-49c9-a71b-c13fcf95dd75 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:31.983063 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c8fb5-3b02-4baf-9dbf-0f92c94457e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:31.983685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c8fb5-3b02-4baf-9dbf-0f92c94457e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:31.983965 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c8fb5-3b02-4baf-9dbf-0f92c94457e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:31.984235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c8fb5-3b02-4baf-9dbf-0f92c94457e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.986975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-883c8fb5-3b02-4baf-9dbf-0f92c94457e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:31.988291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0827f-9db3-4d80-90a3-28dc810bf957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:31.990562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0827f-9db3-4d80-90a3-28dc810bf957 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:31.990562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0827f-9db3-4d80-90a3-28dc810bf957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:31.990562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0827f-9db3-4d80-90a3-28dc810bf957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:31.990562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0827f-9db3-4d80-90a3-28dc810bf957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:31.995618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0827f-9db3-4d80-90a3-28dc810bf957 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:32.007157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 778/1562] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:42:32.011401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0827f-9db3-4d80-90a3-28dc810bf957 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:32.014982 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:32.023286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0827f-9db3-4d80-90a3-28dc810bf957 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:32.030597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c30e71-d064-475d-a292-cf1021c1d2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:32.030855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c30e71-d064-475d-a292-cf1021c1d2ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:32.031089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c30e71-d064-475d-a292-cf1021c1d2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:32.031366 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c30e71-d064-475d-a292-cf1021c1d2ce admin admin] PATH_INFO: `/v3/projects/f9bb360edfae4d37830cd340ae25b59a/users/6501053893ad408db9e647e5ce3b2afe/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:32.048957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c30e71-d064-475d-a292-cf1021c1d2ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9bb360edfae4d37830cd340ae25b59a, user_id=6501053893ad408db9e647e5ce3b2afe, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:32.052096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c30e71-d064-475d-a292-cf1021c1d2ce admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:32.069089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c8fb5-3b02-4baf-9dbf-0f92c94457e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:32.071236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c8fb5-3b02-4baf-9dbf-0f92c94457e2 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:32.074245 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 779/1563] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:32 2022] PUT /identity/v3/projects/f9bb360edfae4d37830cd340ae25b59a/users/6501053893ad408db9e647e5ce3b2afe/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:32.082394 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 785/1564] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:32.086081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b3cc7-3058-4dbd-9edf-6f94af0543d6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:32.086466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b3cc7-3058-4dbd-9edf-6f94af0543d6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:32.086818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b3cc7-3058-4dbd-9edf-6f94af0543d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:32.139382 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-706b3cc7-3058-4dbd-9edf-6f94af0543d6 None None] MFA Rules not processed for user `6501053893ad408db9e647e5ce3b2afe`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:32.140732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-706b3cc7-3058-4dbd-9edf-6f94af0543d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:32.236526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 780/1565] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:32.249149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c8fb5-3b02-4baf-9dbf-0f92c94457e2 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:32.250707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-883c8fb5-3b02-4baf-9dbf-0f92c94457e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:32.264274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c8fb5-3b02-4baf-9dbf-0f92c94457e2 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:32.465029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcffc07-ad24-4589-b389-867e1a467687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:32.465651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcffc07-ad24-4589-b389-867e1a467687 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:32.465929 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcffc07-ad24-4589-b389-867e1a467687 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:32.466863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcffc07-ad24-4589-b389-867e1a467687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:32.476322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcffc07-ad24-4589-b389-867e1a467687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:32.477821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcffc07-ad24-4589-b389-867e1a467687 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:32.487670 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 786/1566] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:40.708106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b3cc7-3058-4dbd-9edf-6f94af0543d6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:40.709673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-706b3cc7-3058-4dbd-9edf-6f94af0543d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:40.739525 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:40.780225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b3cc7-3058-4dbd-9edf-6f94af0543d6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:40.785850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:40.786526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:40.786806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:40.787076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] PATH_INFO: `/v3/users/b2cb2d2f047b46d58d2a05ab0807240d` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:40.788188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2cb2d2f047b46d58d2a05ab0807240d)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:40.790083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:40.836661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2cb2d2f047b46d58d2a05ab0807240d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:40.837314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2cb2d2f047b46d58d2a05ab0807240d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:40.837915 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2cb2d2f047b46d58d2a05ab0807240d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:40.844684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2cb2d2f047b46d58d2a05ab0807240d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:40.856689 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2cb2d2f047b46d58d2a05ab0807240d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:40.869198 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 781/1567] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:40 2022] DELETE /identity/v3/users/b2cb2d2f047b46d58d2a05ab0807240d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:40.873717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53beca2-08d7-455a-80d5-d925c202514a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:40.874241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53beca2-08d7-455a-80d5-d925c202514a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:40.874463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53beca2-08d7-455a-80d5-d925c202514a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:40.894981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c53beca2-08d7-455a-80d5-d925c202514a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:40.896165 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c53beca2-08d7-455a-80d5-d925c202514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:40.955740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 787/1568] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:40.965509 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:40.966916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:41.023938 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737a9f8-c479-4a97-9383-b08c96b5a742 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:41.027621 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1e4e4-4a68-4f84-87af-9774086c71ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:41.028231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1e4e4-4a68-4f84-87af-9774086c71ca service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:41.028509 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1e4e4-4a68-4f84-87af-9774086c71ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:41.028780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1e4e4-4a68-4f84-87af-9774086c71ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:41.038600 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1e4e4-4a68-4f84-87af-9774086c71ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:41.040056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1e4e4-4a68-4f84-87af-9774086c71ca service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:41.050142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 782/1569] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:41.349051 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53beca2-08d7-455a-80d5-d925c202514a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:41.350576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c53beca2-08d7-455a-80d5-d925c202514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:41.365441 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53beca2-08d7-455a-80d5-d925c202514a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:41.370348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:41.371548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:41.371834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:41.372272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] PATH_INFO: `/v3/projects/a08dade49581425d8a4a46d8776ae9f4` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:41.373402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a08dade49581425d8a4a46d8776ae9f4)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:41.375318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:41.422564 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a08dade49581425d8a4a46d8776ae9f4', 'request_id': 'req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:41.428522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a08dade49581425d8a4a46d8776ae9f4', 'request_id': 'req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:41.431012 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:41.436398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a08dade49581425d8a4a46d8776ae9f4', 'request_id': 'req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:41.449871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] The token cache is being invalidate because project a08dade49581425d8a4a46d8776ae9f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:41.450406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:41.451137 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:41.452949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 788/1570] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:41 2022] DELETE /identity/v3/projects/a08dade49581425d8a4a46d8776ae9f4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:41.456913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1e4e4-4a68-4f84-87af-9774086c71ca service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:41.468315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1e4e4-4a68-4f84-87af-9774086c71ca service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:41.513399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:41.529334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d12f60-c3fa-456d-9928-d30044f81495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:41.529975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d12f60-c3fa-456d-9928-d30044f81495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:41.530286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d12f60-c3fa-456d-9928-d30044f81495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:41.530563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d12f60-c3fa-456d-9928-d30044f81495 admin admin] PATH_INFO: `/v3/users/7396ddcc12614b27a4aa26780e40b1a7` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:41.532377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d12f60-c3fa-456d-9928-d30044f81495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7396ddcc12614b27a4aa26780e40b1a7)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:41.535552 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d12f60-c3fa-456d-9928-d30044f81495 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:41.588063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65d12f60-c3fa-456d-9928-d30044f81495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7396ddcc12614b27a4aa26780e40b1a7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:41.588780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65d12f60-c3fa-456d-9928-d30044f81495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7396ddcc12614b27a4aa26780e40b1a7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:41.589423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65d12f60-c3fa-456d-9928-d30044f81495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7396ddcc12614b27a4aa26780e40b1a7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:41.598126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65d12f60-c3fa-456d-9928-d30044f81495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7396ddcc12614b27a4aa26780e40b1a7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:41.617234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65d12f60-c3fa-456d-9928-d30044f81495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7396ddcc12614b27a4aa26780e40b1a7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:41.643742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 783/1571] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:41 2022] DELETE /identity/v3/users/7396ddcc12614b27a4aa26780e40b1a7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:41.992291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:41.994144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:42.007918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bab1a6-9f3d-4e69-a193-5e0e42a8e5a2 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:42.065852 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:42.066517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.066795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.067067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] PATH_INFO: `/v3/projects/e6d7c031d02649f8bf19fe11424dd2b3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.068491 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d7c031d02649f8bf19fe11424dd2b3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:42.070062 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:42.119452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d7c031d02649f8bf19fe11424dd2b3', 'request_id': 'req-326fa0e6-983b-4073-9ab0-d5dfe0411dae'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:42.124933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d7c031d02649f8bf19fe11424dd2b3', 'request_id': 'req-326fa0e6-983b-4073-9ab0-d5dfe0411dae'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:42.131625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d7c031d02649f8bf19fe11424dd2b3', 'request_id': 'req-326fa0e6-983b-4073-9ab0-d5dfe0411dae'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:42.143002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] The token cache is being invalidate because project e6d7c031d02649f8bf19fe11424dd2b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:42.143285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:42.143920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-326fa0e6-983b-4073-9ab0-d5dfe0411dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:42.145390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 789/1572] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:41 2022] DELETE /identity/v3/projects/e6d7c031d02649f8bf19fe11424dd2b3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:42.150291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf304-a053-4e8b-bb81-3fd08c5fb45d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.150743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf304-a053-4e8b-bb81-3fd08c5fb45d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.150878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf304-a053-4e8b-bb81-3fd08c5fb45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.170594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-55ccf304-a053-4e8b-bb81-3fd08c5fb45d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:42.171767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-55ccf304-a053-4e8b-bb81-3fd08c5fb45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:42.230902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 784/1573] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:42.238422 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb05244-c1aa-4e6c-b773-11e49ec90a37 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.238710 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb05244-c1aa-4e6c-b773-11e49ec90a37 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.238985 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb05244-c1aa-4e6c-b773-11e49ec90a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.260568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-afb05244-c1aa-4e6c-b773-11e49ec90a37 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:42.261838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-afb05244-c1aa-4e6c-b773-11e49ec90a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:42.274881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 790/1574] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:42.279171 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccf304-a053-4e8b-bb81-3fd08c5fb45d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:42.289919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccf304-a053-4e8b-bb81-3fd08c5fb45d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:42.294540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e92734-60ab-4bcf-82e3-0c1387fe2155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:42.295144 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e92734-60ab-4bcf-82e3-0c1387fe2155 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.295424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e92734-60ab-4bcf-82e3-0c1387fe2155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.295690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e92734-60ab-4bcf-82e3-0c1387fe2155 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.296334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e92734-60ab-4bcf-82e3-0c1387fe2155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:42.296632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e92734-60ab-4bcf-82e3-0c1387fe2155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:42.298238 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e92734-60ab-4bcf-82e3-0c1387fe2155 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:42.310075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 785/1575] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:42:42.315036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb05244-c1aa-4e6c-b773-11e49ec90a37 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:42.324027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb05244-c1aa-4e6c-b773-11e49ec90a37 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:42.328568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c17b8d-80cb-42e4-85b4-28924aa7129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:42.329165 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c17b8d-80cb-42e4-85b4-28924aa7129f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.329444 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c17b8d-80cb-42e4-85b4-28924aa7129f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.329787 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c17b8d-80cb-42e4-85b4-28924aa7129f admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.330982 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c17b8d-80cb-42e4-85b4-28924aa7129f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:42.332803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c17b8d-80cb-42e4-85b4-28924aa7129f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:42.359912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 791/1576] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:42 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:42.364244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e92734-60ab-4bcf-82e3-0c1387fe2155 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:42.374377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e92734-60ab-4bcf-82e3-0c1387fe2155 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:42.427813 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:42.438251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b4bd9-7a8e-4b36-9944-69d9d213b1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:42.438861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b4bd9-7a8e-4b36-9944-69d9d213b1a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.439656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b4bd9-7a8e-4b36-9944-69d9d213b1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.439941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b4bd9-7a8e-4b36-9944-69d9d213b1a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.440568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b4bd9-7a8e-4b36-9944-69d9d213b1a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:42.442486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b4bd9-7a8e-4b36-9944-69d9d213b1a2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:42.464717 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 786/1577] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:42 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:42.468833 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c17b8d-80cb-42e4-85b4-28924aa7129f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:42.479484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c17b8d-80cb-42e4-85b4-28924aa7129f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:42.485540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94421f3-82d2-4971-b988-70decab61dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:42.486114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94421f3-82d2-4971-b988-70decab61dfa admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.486421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94421f3-82d2-4971-b988-70decab61dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.486688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94421f3-82d2-4971-b988-70decab61dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.487325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94421f3-82d2-4971-b988-70decab61dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:42.488890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94421f3-82d2-4971-b988-70decab61dfa admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:42.499526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 792/1578] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:42.504059 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b4bd9-7a8e-4b36-9944-69d9d213b1a2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:42.514108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b4bd9-7a8e-4b36-9944-69d9d213b1a2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:42.519542 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a438bf-69ab-4e8b-ae93-6b2c9070c61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:42.520172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a438bf-69ab-4e8b-ae93-6b2c9070c61c admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.520452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a438bf-69ab-4e8b-ae93-6b2c9070c61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.520740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a438bf-69ab-4e8b-ae93-6b2c9070c61c admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.521404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a438bf-69ab-4e8b-ae93-6b2c9070c61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:42.523053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a438bf-69ab-4e8b-ae93-6b2c9070c61c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:42.533788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 787/1579] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:42.537909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94421f3-82d2-4971-b988-70decab61dfa admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:42.547924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94421f3-82d2-4971-b988-70decab61dfa admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:42.552576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad581065-aa64-4943-8d58-68b6473639d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:42.553248 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad581065-aa64-4943-8d58-68b6473639d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.553528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad581065-aa64-4943-8d58-68b6473639d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.553800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad581065-aa64-4943-8d58-68b6473639d8 admin admin] PATH_INFO: `/v3/projects/dc2f7511315d4c339daf24948ff28226/users/cde41d56c4a24c469d6409544abe8635/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.568683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad581065-aa64-4943-8d58-68b6473639d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2f7511315d4c339daf24948ff28226, user_id=cde41d56c4a24c469d6409544abe8635, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:42.571338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad581065-aa64-4943-8d58-68b6473639d8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:42.584376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 793/1580] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:42 2022] PUT /identity/v3/projects/dc2f7511315d4c339daf24948ff28226/users/cde41d56c4a24c469d6409544abe8635/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:42.594585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c40242-e015-412d-83ee-98f1efb27f36 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.594871 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c40242-e015-412d-83ee-98f1efb27f36 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.595144 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c40242-e015-412d-83ee-98f1efb27f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.622118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-95c40242-e015-412d-83ee-98f1efb27f36 None None] MFA Rules not processed for user `cde41d56c4a24c469d6409544abe8635`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:42.623346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-95c40242-e015-412d-83ee-98f1efb27f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:42.688850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 788/1581] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:42.694305 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad581065-aa64-4943-8d58-68b6473639d8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:42.712377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:42.769627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad581065-aa64-4943-8d58-68b6473639d8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:42.774866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa92876-b58a-4a46-ac73-3bd1217754a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:42.775483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa92876-b58a-4a46-ac73-3bd1217754a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.775885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa92876-b58a-4a46-ac73-3bd1217754a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.776007 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa92876-b58a-4a46-ac73-3bd1217754a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.776987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa92876-b58a-4a46-ac73-3bd1217754a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:42.778586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa92876-b58a-4a46-ac73-3bd1217754a6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:42.813095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 794/1582] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:42 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:42.814985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c40242-e015-412d-83ee-98f1efb27f36 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:42.827997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c40242-e015-412d-83ee-98f1efb27f36 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:42.886363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96034c2-486b-4f28-96f6-2429b93b03ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:42.886979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96034c2-486b-4f28-96f6-2429b93b03ac admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.887262 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96034c2-486b-4f28-96f6-2429b93b03ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.887533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96034c2-486b-4f28-96f6-2429b93b03ac admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.888162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96034c2-486b-4f28-96f6-2429b93b03ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:42.890065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96034c2-486b-4f28-96f6-2429b93b03ac admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:42.912113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 789/1583] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:42 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:42.916123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa92876-b58a-4a46-ac73-3bd1217754a6 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:42.927000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa92876-b58a-4a46-ac73-3bd1217754a6 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:42.931553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd8075-f43e-4441-9c7d-3d6b6bfa7be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:42.932149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd8075-f43e-4441-9c7d-3d6b6bfa7be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.932428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd8075-f43e-4441-9c7d-3d6b6bfa7be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.932700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd8075-f43e-4441-9c7d-3d6b6bfa7be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.933331 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd8075-f43e-4441-9c7d-3d6b6bfa7be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:42.934948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd8075-f43e-4441-9c7d-3d6b6bfa7be2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:42.945210 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 795/1584] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:42.949126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96034c2-486b-4f28-96f6-2429b93b03ac admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:42.959050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96034c2-486b-4f28-96f6-2429b93b03ac admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:42.963628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0b6f6-198f-4b1a-9b91-816846c96bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:42.964273 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0b6f6-198f-4b1a-9b91-816846c96bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:42.964555 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0b6f6-198f-4b1a-9b91-816846c96bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:42.964827 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0b6f6-198f-4b1a-9b91-816846c96bd3 admin admin] PATH_INFO: `/v3/projects/681035c12e0c45b6ade5708c3d657cf1/users/dbd121fa77f3419b82d786b9bf4d6859/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:42.979344 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0b6f6-198f-4b1a-9b91-816846c96bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681035c12e0c45b6ade5708c3d657cf1, user_id=dbd121fa77f3419b82d786b9bf4d6859, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:42.982039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0b6f6-198f-4b1a-9b91-816846c96bd3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:42.995777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 790/1585] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:42 2022] PUT /identity/v3/projects/681035c12e0c45b6ade5708c3d657cf1/users/dbd121fa77f3419b82d786b9bf4d6859/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:42.999507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd8075-f43e-4441-9c7d-3d6b6bfa7be2 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:43.009960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd8075-f43e-4441-9c7d-3d6b6bfa7be2 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:43.071805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2168f4e-f238-4483-ac44-3edfb80cd049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:43.072416 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2168f4e-f238-4483-ac44-3edfb80cd049 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:43.072830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2168f4e-f238-4483-ac44-3edfb80cd049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:43.073108 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2168f4e-f238-4483-ac44-3edfb80cd049 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:43.073749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2168f4e-f238-4483-ac44-3edfb80cd049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:43.075364 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2168f4e-f238-4483-ac44-3edfb80cd049 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:43.085855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 796/1586] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:43.090113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0b6f6-198f-4b1a-9b91-816846c96bd3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:43.101468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0b6f6-198f-4b1a-9b91-816846c96bd3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:43.106709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a074f1b-a7c4-4d29-8248-ef652cbf16dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:43.107369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a074f1b-a7c4-4d29-8248-ef652cbf16dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:43.107651 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a074f1b-a7c4-4d29-8248-ef652cbf16dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:43.107925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a074f1b-a7c4-4d29-8248-ef652cbf16dc admin admin] PATH_INFO: `/v3/projects/681035c12e0c45b6ade5708c3d657cf1/users/dbd121fa77f3419b82d786b9bf4d6859/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:43.109831 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a074f1b-a7c4-4d29-8248-ef652cbf16dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=681035c12e0c45b6ade5708c3d657cf1, user_id=dbd121fa77f3419b82d786b9bf4d6859, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:43.113386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a074f1b-a7c4-4d29-8248-ef652cbf16dc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:43.128731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 791/1587] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:43 2022] PUT /identity/v3/projects/681035c12e0c45b6ade5708c3d657cf1/users/dbd121fa77f3419b82d786b9bf4d6859/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:43.136091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d9fad-8c0f-44f5-8f8b-a643efc92d63 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:43.136363 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d9fad-8c0f-44f5-8f8b-a643efc92d63 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:43.136642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d9fad-8c0f-44f5-8f8b-a643efc92d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:43.169317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4e6d9fad-8c0f-44f5-8f8b-a643efc92d63 None None] MFA Rules not processed for user `dbd121fa77f3419b82d786b9bf4d6859`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:43.169317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4e6d9fad-8c0f-44f5-8f8b-a643efc92d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:43.240012 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 797/1588] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:43.250907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a074f1b-a7c4-4d29-8248-ef652cbf16dc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:43.252645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1a074f1b-a7c4-4d29-8248-ef652cbf16dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:43.268841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:43.314872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a074f1b-a7c4-4d29-8248-ef652cbf16dc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:43.318569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ba5e1-5773-4630-ac37-c14e8aa3de3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:43.319180 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ba5e1-5773-4630-ac37-c14e8aa3de3e service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:43.319461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ba5e1-5773-4630-ac37-c14e8aa3de3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:43.319732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ba5e1-5773-4630-ac37-c14e8aa3de3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:43.328580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ba5e1-5773-4630-ac37-c14e8aa3de3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:43.330104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ba5e1-5773-4630-ac37-c14e8aa3de3e service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:43.339477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 792/1589] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:43.444434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d9fad-8c0f-44f5-8f8b-a643efc92d63 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:43.453862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d9fad-8c0f-44f5-8f8b-a643efc92d63 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:43.457980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafeb0f8-b6a0-4a84-a2a7-4b1967d8f637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:43.458624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafeb0f8-b6a0-4a84-a2a7-4b1967d8f637 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:43.458906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafeb0f8-b6a0-4a84-a2a7-4b1967d8f637 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:43.459180 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafeb0f8-b6a0-4a84-a2a7-4b1967d8f637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:43.467795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:43.520586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafeb0f8-b6a0-4a84-a2a7-4b1967d8f637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:43.522645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafeb0f8-b6a0-4a84-a2a7-4b1967d8f637 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:43.532455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 798/1590] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:52.119156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ba5e1-5773-4630-ac37-c14e8aa3de3e service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:52.123199 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-139ba5e1-5773-4630-ac37-c14e8aa3de3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:52.149016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ba5e1-5773-4630-ac37-c14e8aa3de3e service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:52.155356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb3fd8-d5fd-4a6b-ad5b-2893c76236fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:52.159557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb3fd8-d5fd-4a6b-ad5b-2893c76236fc service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:52.159841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb3fd8-d5fd-4a6b-ad5b-2893c76236fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:52.160218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb3fd8-d5fd-4a6b-ad5b-2893c76236fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:52.163147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-affb3fd8-d5fd-4a6b-ad5b-2893c76236fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:52.243231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb3fd8-d5fd-4a6b-ad5b-2893c76236fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:52.244670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb3fd8-d5fd-4a6b-ad5b-2893c76236fc service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:52.255184 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 793/1591] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:52.491092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafeb0f8-b6a0-4a84-a2a7-4b1967d8f637 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:52.572796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafeb0f8-b6a0-4a84-a2a7-4b1967d8f637 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:52.584114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:52.584619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:52.584897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:52.585167 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] PATH_INFO: `/v3/users/cde41d56c4a24c469d6409544abe8635` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:52.586563 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cde41d56c4a24c469d6409544abe8635)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:52.589310 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:52.685869 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cde41d56c4a24c469d6409544abe8635'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:52.686825 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cde41d56c4a24c469d6409544abe8635'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:52.707681 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cde41d56c4a24c469d6409544abe8635'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:52.708878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cde41d56c4a24c469d6409544abe8635'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:52.719957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cde41d56c4a24c469d6409544abe8635'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:52.738273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 799/1592] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:52 2022] DELETE /identity/v3/users/cde41d56c4a24c469d6409544abe8635 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:52.743168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26d092-0d69-49c7-bc32-33132764e3e3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:52.744172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26d092-0d69-49c7-bc32-33132764e3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:52.744801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26d092-0d69-49c7-bc32-33132764e3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:52.769711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4b26d092-0d69-49c7-bc32-33132764e3e3 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:52.771115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4b26d092-0d69-49c7-bc32-33132764e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:52.847666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 794/1593] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:52.862413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:52.862413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:52.896360 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:52.956745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ca0e5-3788-47e5-99b0-30c35d4a03f9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:52.961153 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448423a-d0d2-4389-8b66-bb00c7b843b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:52.961768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448423a-d0d2-4389-8b66-bb00c7b843b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:52.962039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448423a-d0d2-4389-8b66-bb00c7b843b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:52.962352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448423a-d0d2-4389-8b66-bb00c7b843b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:52.979864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448423a-d0d2-4389-8b66-bb00c7b843b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:52.981433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448423a-d0d2-4389-8b66-bb00c7b843b6 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:52.992971 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 800/1594] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:53.347231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26d092-0d69-49c7-bc32-33132764e3e3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:53.348798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4b26d092-0d69-49c7-bc32-33132764e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:53.372018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26d092-0d69-49c7-bc32-33132764e3e3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:53.411389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:53.412170 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:53.412450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:53.412724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] PATH_INFO: `/v3/projects/dc2f7511315d4c339daf24948ff28226` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:53.413901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc2f7511315d4c339daf24948ff28226)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:53.416117 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:53.486696 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc2f7511315d4c339daf24948ff28226', 'request_id': 'req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:53.489065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:53.499458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc2f7511315d4c339daf24948ff28226', 'request_id': 'req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:53.507921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc2f7511315d4c339daf24948ff28226', 'request_id': 'req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:53.519409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] The token cache is being invalidate because project dc2f7511315d4c339daf24948ff28226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:53.519701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:53.520362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:53.521865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 795/1595] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:53 2022] DELETE /identity/v3/projects/dc2f7511315d4c339daf24948ff28226 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:53.525881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448423a-d0d2-4389-8b66-bb00c7b843b6 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:53.537081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448423a-d0d2-4389-8b66-bb00c7b843b6 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:53.592495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57f8e8-939b-4e7d-90fc-50a1ccb2dce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:53.593116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57f8e8-939b-4e7d-90fc-50a1ccb2dce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:53.593397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57f8e8-939b-4e7d-90fc-50a1ccb2dce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:53.593668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57f8e8-939b-4e7d-90fc-50a1ccb2dce3 admin admin] PATH_INFO: `/v3/users/dbd121fa77f3419b82d786b9bf4d6859` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:53.594765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57f8e8-939b-4e7d-90fc-50a1ccb2dce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbd121fa77f3419b82d786b9bf4d6859)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:53.596719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57f8e8-939b-4e7d-90fc-50a1ccb2dce3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:53.648217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-db57f8e8-939b-4e7d-90fc-50a1ccb2dce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbd121fa77f3419b82d786b9bf4d6859'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:53.649009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-db57f8e8-939b-4e7d-90fc-50a1ccb2dce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbd121fa77f3419b82d786b9bf4d6859'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:53.662550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-db57f8e8-939b-4e7d-90fc-50a1ccb2dce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbd121fa77f3419b82d786b9bf4d6859'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:53.663231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-db57f8e8-939b-4e7d-90fc-50a1ccb2dce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbd121fa77f3419b82d786b9bf4d6859'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:53.670341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-db57f8e8-939b-4e7d-90fc-50a1ccb2dce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbd121fa77f3419b82d786b9bf4d6859'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:53.684094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 801/1596] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:53 2022] DELETE /identity/v3/users/dbd121fa77f3419b82d786b9bf4d6859 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:54.009610 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:54.011076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:54.052530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a3a2e-ab97-433c-a164-b3c6334b8b55 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:54.193115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:54.193768 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:54.194054 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:54.194357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] PATH_INFO: `/v3/projects/681035c12e0c45b6ade5708c3d657cf1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:54.213735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=681035c12e0c45b6ade5708c3d657cf1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:54.216086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:54.301406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681035c12e0c45b6ade5708c3d657cf1', 'request_id': 'req-633f3dca-fa18-4546-b2c4-827831972c86'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:54.324187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681035c12e0c45b6ade5708c3d657cf1', 'request_id': 'req-633f3dca-fa18-4546-b2c4-827831972c86'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:54.351687 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681035c12e0c45b6ade5708c3d657cf1', 'request_id': 'req-633f3dca-fa18-4546-b2c4-827831972c86'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:54.365470 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] The token cache is being invalidate because project 681035c12e0c45b6ade5708c3d657cf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:54.365811 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:54.366503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-633f3dca-fa18-4546-b2c4-827831972c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:54.368054 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 796/1597] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:54 2022] DELETE /identity/v3/projects/681035c12e0c45b6ade5708c3d657cf1 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:54.372792 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35449bc-df67-4bf5-bd72-e9c752972773 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:54.373078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35449bc-df67-4bf5-bd72-e9c752972773 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:54.373336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35449bc-df67-4bf5-bd72-e9c752972773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:54.393792 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b35449bc-df67-4bf5-bd72-e9c752972773 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:54.395145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b35449bc-df67-4bf5-bd72-e9c752972773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:54.448149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:54.467436 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 802/1598] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:54.475576 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd413c-00b9-4062-bdb6-302fe605966e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:54.475966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd413c-00b9-4062-bdb6-302fe605966e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:54.476144 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd413c-00b9-4062-bdb6-302fe605966e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:54.496335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a0dd413c-00b9-4062-bdb6-302fe605966e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:54.497532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a0dd413c-00b9-4062-bdb6-302fe605966e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:54.511228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 797/1599] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:54.511746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35449bc-df67-4bf5-bd72-e9c752972773 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:54.527149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35449bc-df67-4bf5-bd72-e9c752972773 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:54.532320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7a5dd-6eaf-480a-8857-8bf550e7cddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:54.533922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7a5dd-6eaf-480a-8857-8bf550e7cddd admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:54.534314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7a5dd-6eaf-480a-8857-8bf550e7cddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:54.534686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7a5dd-6eaf-480a-8857-8bf550e7cddd admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:54.535336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7a5dd-6eaf-480a-8857-8bf550e7cddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:54.535689 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7a5dd-6eaf-480a-8857-8bf550e7cddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:54.537665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7a5dd-6eaf-480a-8857-8bf550e7cddd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:54.563904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 803/1600] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:42:54.569482 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd413c-00b9-4062-bdb6-302fe605966e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:54.587362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd413c-00b9-4062-bdb6-302fe605966e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:54.600255 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289cbfb7-f242-4ddf-8c2a-006acc413b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:54.600986 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cbfb7-f242-4ddf-8c2a-006acc413b52 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:54.601524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cbfb7-f242-4ddf-8c2a-006acc413b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:54.601862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cbfb7-f242-4ddf-8c2a-006acc413b52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:54.602612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289cbfb7-f242-4ddf-8c2a-006acc413b52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:54.604724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289cbfb7-f242-4ddf-8c2a-006acc413b52 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:54.675000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 798/1601] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:54 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:42:54.679539 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7a5dd-6eaf-480a-8857-8bf550e7cddd admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:54.692046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7a5dd-6eaf-480a-8857-8bf550e7cddd admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:54.773155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0c3be-b2bb-4e14-abf9-81fafb964e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:54.778944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0c3be-b2bb-4e14-abf9-81fafb964e3c admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:54.779231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0c3be-b2bb-4e14-abf9-81fafb964e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:54.779543 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0c3be-b2bb-4e14-abf9-81fafb964e3c admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:54.780145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0c3be-b2bb-4e14-abf9-81fafb964e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:54.782716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0c3be-b2bb-4e14-abf9-81fafb964e3c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:54.809376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 804/1602] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:54 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:54.810442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289cbfb7-f242-4ddf-8c2a-006acc413b52 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:54.821799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289cbfb7-f242-4ddf-8c2a-006acc413b52 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:54.826889 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670c270-0b6b-4b90-bcb4-f1f22357b0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:54.827575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670c270-0b6b-4b90-bcb4-f1f22357b0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:54.827830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670c270-0b6b-4b90-bcb4-f1f22357b0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:54.828314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670c270-0b6b-4b90-bcb4-f1f22357b0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:54.829100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670c270-0b6b-4b90-bcb4-f1f22357b0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:54.831293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670c270-0b6b-4b90-bcb4-f1f22357b0b2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:54.843484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 799/1603] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:54.847678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670c270-0b6b-4b90-bcb4-f1f22357b0b2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:54.847853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0c3be-b2bb-4e14-abf9-81fafb964e3c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:54.849459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f1a0c3be-b2bb-4e14-abf9-81fafb964e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:54.858391 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670c270-0b6b-4b90-bcb4-f1f22357b0b2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:54.864838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accbab2-dfd0-4961-9987-1f5be9d3a502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:54.865359 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0c3be-b2bb-4e14-abf9-81fafb964e3c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:54.865600 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accbab2-dfd0-4961-9987-1f5be9d3a502 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:54.865983 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accbab2-dfd0-4961-9987-1f5be9d3a502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:54.866273 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7accbab2-dfd0-4961-9987-1f5be9d3a502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:54.867233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accbab2-dfd0-4961-9987-1f5be9d3a502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:54.869261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7accbab2-dfd0-4961-9987-1f5be9d3a502 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:54.873456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d479ca-c5f3-4e6c-a618-72ecf44b181c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:54.874269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d479ca-c5f3-4e6c-a618-72ecf44b181c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:54.874558 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d479ca-c5f3-4e6c-a618-72ecf44b181c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:54.874830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d479ca-c5f3-4e6c-a618-72ecf44b181c admin admin] PATH_INFO: `/v3/users/8596a908ae40431c8fa25c11b6ac9c7b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:54.876633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d479ca-c5f3-4e6c-a618-72ecf44b181c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8596a908ae40431c8fa25c11b6ac9c7b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:54.879830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d479ca-c5f3-4e6c-a618-72ecf44b181c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:54.883121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 800/1604] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:54.887214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accbab2-dfd0-4961-9987-1f5be9d3a502 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:54.900567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7accbab2-dfd0-4961-9987-1f5be9d3a502 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:54.906072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3c023-7ecc-4b12-b23c-0873d3fc55b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:54.907018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3c023-7ecc-4b12-b23c-0873d3fc55b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:54.907444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3c023-7ecc-4b12-b23c-0873d3fc55b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:54.908093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3c023-7ecc-4b12-b23c-0873d3fc55b3 admin admin] PATH_INFO: `/v3/projects/5a91009b85ad4d1195ed75d0bfe853de/users/d317698f009d40f89953c40969083f8d/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:54.931250 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3c023-7ecc-4b12-b23c-0873d3fc55b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a91009b85ad4d1195ed75d0bfe853de, user_id=d317698f009d40f89953c40969083f8d, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:54.934689 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3c023-7ecc-4b12-b23c-0873d3fc55b3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:54.937683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:54.956628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 801/1605] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:54 2022] PUT /identity/v3/projects/5a91009b85ad4d1195ed75d0bfe853de/users/d317698f009d40f89953c40969083f8d/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:54.966931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e105e-7534-44cf-997f-63addfea4b1e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:54.967660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e105e-7534-44cf-997f-63addfea4b1e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:54.968120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e105e-7534-44cf-997f-63addfea4b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:54.968314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-40d479ca-c5f3-4e6c-a618-72ecf44b181c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8596a908ae40431c8fa25c11b6ac9c7b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:54.969054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-40d479ca-c5f3-4e6c-a618-72ecf44b181c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8596a908ae40431c8fa25c11b6ac9c7b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:54.989044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-40d479ca-c5f3-4e6c-a618-72ecf44b181c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8596a908ae40431c8fa25c11b6ac9c7b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:54.990132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-40d479ca-c5f3-4e6c-a618-72ecf44b181c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8596a908ae40431c8fa25c11b6ac9c7b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:55.004011 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-032e105e-7534-44cf-997f-63addfea4b1e None None] MFA Rules not processed for user `d317698f009d40f89953c40969083f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:55.005373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-40d479ca-c5f3-4e6c-a618-72ecf44b181c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8596a908ae40431c8fa25c11b6ac9c7b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:55.005935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-032e105e-7534-44cf-997f-63addfea4b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:55.025184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 805/1606] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:54 2022] DELETE /identity/v3/users/8596a908ae40431c8fa25c11b6ac9c7b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:55.034158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5f9d0-b635-4710-96fd-f3d965c7b158 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.034518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5f9d0-b635-4710-96fd-f3d965c7b158 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.034816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5f9d0-b635-4710-96fd-f3d965c7b158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.072857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-22e5f9d0-b635-4710-96fd-f3d965c7b158 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:55.074876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-22e5f9d0-b635-4710-96fd-f3d965c7b158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:55.128017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 802/1607] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:55.131906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:55.132999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e105e-7534-44cf-997f-63addfea4b1e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:55.134844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-032e105e-7534-44cf-997f-63addfea4b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:55.166446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 806/1608] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:42:55.174635 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5f9d0-b635-4710-96fd-f3d965c7b158 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:55.176039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-22e5f9d0-b635-4710-96fd-f3d965c7b158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:55.221967 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e105e-7534-44cf-997f-63addfea4b1e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:55.229484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b7ecc-55db-4c96-8519-9b3aa99be071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:55.230192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7ecc-55db-4c96-8519-9b3aa99be071 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.230657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7ecc-55db-4c96-8519-9b3aa99be071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.230997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b7ecc-55db-4c96-8519-9b3aa99be071 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.231789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b7ecc-55db-4c96-8519-9b3aa99be071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:55.233736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b7ecc-55db-4c96-8519-9b3aa99be071 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:55.245012 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5f9d0-b635-4710-96fd-f3d965c7b158 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:55.248882 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72aca7-5c81-4f71-976c-823cb1de8f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:55.249495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72aca7-5c81-4f71-976c-823cb1de8f25 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.249773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72aca7-5c81-4f71-976c-823cb1de8f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.250040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72aca7-5c81-4f71-976c-823cb1de8f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.261212 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72aca7-5c81-4f71-976c-823cb1de8f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:55.264935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 803/1609] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:55 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:42:55.265060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72aca7-5c81-4f71-976c-823cb1de8f25 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:55.265700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b7ecc-55db-4c96-8519-9b3aa99be071 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:55.274111 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 807/1610] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:42:55.339176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b7ecc-55db-4c96-8519-9b3aa99be071 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:55.344895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6bf9f-fab0-4cb8-b1d2-6a18e97fdfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:55.345466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6bf9f-fab0-4cb8-b1d2-6a18e97fdfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.345881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6bf9f-fab0-4cb8-b1d2-6a18e97fdfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.346153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6bf9f-fab0-4cb8-b1d2-6a18e97fdfc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.347360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6bf9f-fab0-4cb8-b1d2-6a18e97fdfc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:55.348934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6bf9f-fab0-4cb8-b1d2-6a18e97fdfc5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:55.368459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:55.379159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 804/1611] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:55 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:42:55.380013 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72aca7-5c81-4f71-976c-823cb1de8f25 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:55.390376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72aca7-5c81-4f71-976c-823cb1de8f25 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:55.396029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7f90e-e1bc-47af-a7c1-6b3d52f1a950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:55.397070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7f90e-e1bc-47af-a7c1-6b3d52f1a950 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.398398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7f90e-e1bc-47af-a7c1-6b3d52f1a950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.398691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7f90e-e1bc-47af-a7c1-6b3d52f1a950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.399462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7f90e-e1bc-47af-a7c1-6b3d52f1a950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:55.402442 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7f90e-e1bc-47af-a7c1-6b3d52f1a950 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:55.417401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 808/1612] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:42:55.419027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6bf9f-fab0-4cb8-b1d2-6a18e97fdfc5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:55.437254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6bf9f-fab0-4cb8-b1d2-6a18e97fdfc5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:55.443083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba97910-4f1e-4139-baf6-0b7b917ba5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:55.443855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba97910-4f1e-4139-baf6-0b7b917ba5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.444205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba97910-4f1e-4139-baf6-0b7b917ba5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.444550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba97910-4f1e-4139-baf6-0b7b917ba5d7 admin admin] PATH_INFO: `/v3/projects/d36ab2a3e2d14349b2bea21370c91da3/users/32365633b27845d09d7f13f221e7a1e8/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.460772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba97910-4f1e-4139-baf6-0b7b917ba5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d36ab2a3e2d14349b2bea21370c91da3, user_id=32365633b27845d09d7f13f221e7a1e8, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:55.463576 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba97910-4f1e-4139-baf6-0b7b917ba5d7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:55.479533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 805/1613] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:55 2022] PUT /identity/v3/projects/d36ab2a3e2d14349b2bea21370c91da3/users/32365633b27845d09d7f13f221e7a1e8/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:55.483619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7f90e-e1bc-47af-a7c1-6b3d52f1a950 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:55.497783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7f90e-e1bc-47af-a7c1-6b3d52f1a950 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:55.602975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50b9f2-e714-4055-b91f-ef949c56a7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:55.603652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50b9f2-e714-4055-b91f-ef949c56a7bc admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.604221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50b9f2-e714-4055-b91f-ef949c56a7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.604221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50b9f2-e714-4055-b91f-ef949c56a7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.604750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50b9f2-e714-4055-b91f-ef949c56a7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:55.606414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50b9f2-e714-4055-b91f-ef949c56a7bc admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:55.621614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 809/1614] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:42:55.622636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba97910-4f1e-4139-baf6-0b7b917ba5d7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:55.634412 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba97910-4f1e-4139-baf6-0b7b917ba5d7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:55.639796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15a03c-e230-4320-a264-892e4c917a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:55.640801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15a03c-e230-4320-a264-892e4c917a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.641781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15a03c-e230-4320-a264-892e4c917a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.643348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15a03c-e230-4320-a264-892e4c917a9d admin admin] PATH_INFO: `/v3/projects/d36ab2a3e2d14349b2bea21370c91da3/users/32365633b27845d09d7f13f221e7a1e8/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.645637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15a03c-e230-4320-a264-892e4c917a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d36ab2a3e2d14349b2bea21370c91da3, user_id=32365633b27845d09d7f13f221e7a1e8, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:55.648377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15a03c-e230-4320-a264-892e4c917a9d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:55.662845 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 806/1615] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:55 2022] PUT /identity/v3/projects/d36ab2a3e2d14349b2bea21370c91da3/users/32365633b27845d09d7f13f221e7a1e8/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:55.674627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7df1a4-419a-4920-8edf-a95b901263c4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.674970 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7df1a4-419a-4920-8edf-a95b901263c4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.675208 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7df1a4-419a-4920-8edf-a95b901263c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.689093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15a03c-e230-4320-a264-892e4c917a9d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:55.691318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5c15a03c-e230-4320-a264-892e4c917a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:55.706296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-2c7df1a4-419a-4920-8edf-a95b901263c4 None None] MFA Rules not processed for user `32365633b27845d09d7f13f221e7a1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:55.707651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2c7df1a4-419a-4920-8edf-a95b901263c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:55.764629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15a03c-e230-4320-a264-892e4c917a9d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:55.769871 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:55.770019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:55.770599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.770882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.771157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] PATH_INFO: `/v3/projects/3c91e8ff0bfc4f78a3165c3583c5085a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.772527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c91e8ff0bfc4f78a3165c3583c5085a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:55.774404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:55.781032 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 810/1616] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:55.792704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7df1a4-419a-4920-8edf-a95b901263c4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:55.794399 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2c7df1a4-419a-4920-8edf-a95b901263c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:55.832775 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c91e8ff0bfc4f78a3165c3583c5085a', 'request_id': 'req-6ea1940f-a4b5-4024-a57c-07852e1aefc9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:55.846786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c91e8ff0bfc4f78a3165c3583c5085a', 'request_id': 'req-6ea1940f-a4b5-4024-a57c-07852e1aefc9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:55.856005 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c91e8ff0bfc4f78a3165c3583c5085a', 'request_id': 'req-6ea1940f-a4b5-4024-a57c-07852e1aefc9'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:55.869687 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] The token cache is being invalidate because project 3c91e8ff0bfc4f78a3165c3583c5085a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:55.869993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:55.870697 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:55.871652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7df1a4-419a-4920-8edf-a95b901263c4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:55.872318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 807/1617] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:55 2022] DELETE /identity/v3/projects/3c91e8ff0bfc4f78a3165c3583c5085a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:55.877136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:55.878670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:55.943654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703c382-581d-4d8b-8748-578f546fe25a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:55.944856 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703c382-581d-4d8b-8748-578f546fe25a service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.945260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703c382-581d-4d8b-8748-578f546fe25a service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.945848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703c382-581d-4d8b-8748-578f546fe25a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.949362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f703c382-581d-4d8b-8748-578f546fe25a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:55.958944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:55.984016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1940f-a4b5-4024-a57c-07852e1aefc9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:55.991672 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:55.992120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:55.992465 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:55.992805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] PATH_INFO: `/v3/users/30ccb34c05fc4faaa63ec1249ed031ec` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:55.994076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ccb34c05fc4faaa63ec1249ed031ec)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:55.996086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:56.042075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703c382-581d-4d8b-8748-578f546fe25a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:56.043481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703c382-581d-4d8b-8748-578f546fe25a service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:56.046931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ccb34c05fc4faaa63ec1249ed031ec'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.047675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ccb34c05fc4faaa63ec1249ed031ec'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.049036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ccb34c05fc4faaa63ec1249ed031ec'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.052755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 811/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:42:56.056392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ccb34c05fc4faaa63ec1249ed031ec'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.069422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ccb34c05fc4faaa63ec1249ed031ec'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.084695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 808/1619] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:55 2022] DELETE /identity/v3/users/30ccb34c05fc4faaa63ec1249ed031ec => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:56.375975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703c382-581d-4d8b-8748-578f546fe25a service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:56.407242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703c382-581d-4d8b-8748-578f546fe25a service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:56.497862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:56.498449 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:56.498732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:56.499054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] PATH_INFO: `/v3/projects/ebfbb5b488f54d75ac0cbb0965ad61a3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:56.500166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebfbb5b488f54d75ac0cbb0965ad61a3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:56.502092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:56.508559 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:56.553096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebfbb5b488f54d75ac0cbb0965ad61a3', 'request_id': 'req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.558571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebfbb5b488f54d75ac0cbb0965ad61a3', 'request_id': 'req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.575374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebfbb5b488f54d75ac0cbb0965ad61a3', 'request_id': 'req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.589689 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] The token cache is being invalidate because project ebfbb5b488f54d75ac0cbb0965ad61a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:56.589976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.590652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.592209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 812/1620] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:56 2022] DELETE /identity/v3/projects/ebfbb5b488f54d75ac0cbb0965ad61a3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:56.596946 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:56.598779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:56.614063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f16fe-7e67-41bd-80c8-65824803fd72 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:56.703964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ae5aa-0b89-48f8-88b3-8520dc105ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:56.704620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ae5aa-0b89-48f8-88b3-8520dc105ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:56.705448 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ae5aa-0b89-48f8-88b3-8520dc105ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:56.706611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ae5aa-0b89-48f8-88b3-8520dc105ddd admin admin] PATH_INFO: `/v3/users/f554685b8ce54c5495ace9e470231c30` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:56.710276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ae5aa-0b89-48f8-88b3-8520dc105ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f554685b8ce54c5495ace9e470231c30)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:56.712823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ae5aa-0b89-48f8-88b3-8520dc105ddd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:56.791118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d4ae5aa-0b89-48f8-88b3-8520dc105ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f554685b8ce54c5495ace9e470231c30'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.791810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d4ae5aa-0b89-48f8-88b3-8520dc105ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f554685b8ce54c5495ace9e470231c30'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.792423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d4ae5aa-0b89-48f8-88b3-8520dc105ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f554685b8ce54c5495ace9e470231c30'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.799966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d4ae5aa-0b89-48f8-88b3-8520dc105ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f554685b8ce54c5495ace9e470231c30'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.812589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:56.815591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d4ae5aa-0b89-48f8-88b3-8520dc105ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f554685b8ce54c5495ace9e470231c30'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:56.831067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 809/1621] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:56 2022] DELETE /identity/v3/users/f554685b8ce54c5495ace9e470231c30 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:57.180677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:57.182146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:57.195392 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc89a3-487c-4ab3-845c-41ccc43e60a7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:57.271557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:57.272193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:57.272471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:57.272739 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] PATH_INFO: `/v3/projects/c4b72c177620487f803307ca7ff4f0c1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:57.273877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b72c177620487f803307ca7ff4f0c1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:57.275859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:57.322067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b72c177620487f803307ca7ff4f0c1', 'request_id': 'req-979d3926-7079-47d2-a896-1ceaf71a75cb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:57.327624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b72c177620487f803307ca7ff4f0c1', 'request_id': 'req-979d3926-7079-47d2-a896-1ceaf71a75cb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:57.334854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b72c177620487f803307ca7ff4f0c1', 'request_id': 'req-979d3926-7079-47d2-a896-1ceaf71a75cb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:57.346151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] The token cache is being invalidate because project c4b72c177620487f803307ca7ff4f0c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:57.346466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:57.347095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-979d3926-7079-47d2-a896-1ceaf71a75cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:57.348577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 813/1622] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:57 2022] DELETE /identity/v3/projects/c4b72c177620487f803307ca7ff4f0c1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:57.354542 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366d28e-7af4-4423-8318-71f90804c7fc None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:57.354835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366d28e-7af4-4423-8318-71f90804c7fc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:57.355133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366d28e-7af4-4423-8318-71f90804c7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:57.375269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8366d28e-7af4-4423-8318-71f90804c7fc None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:57.376469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8366d28e-7af4-4423-8318-71f90804c7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:57.436516 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 810/1623] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:57.445484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9b3db-9fd0-4663-82c3-133ce4052cc8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:57.445774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9b3db-9fd0-4663-82c3-133ce4052cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:57.446049 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9b3db-9fd0-4663-82c3-133ce4052cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:57.465984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0ec9b3db-9fd0-4663-82c3-133ce4052cc8 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:57.467284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0ec9b3db-9fd0-4663-82c3-133ce4052cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:57.477726 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 814/1624] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:57.482103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366d28e-7af4-4423-8318-71f90804c7fc None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:57.492756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366d28e-7af4-4423-8318-71f90804c7fc None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:57.497592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a0675-662d-43f6-aa2d-5e229990c2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:57.498096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a0675-662d-43f6-aa2d-5e229990c2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:57.498405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a0675-662d-43f6-aa2d-5e229990c2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:57.498679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a0675-662d-43f6-aa2d-5e229990c2e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:57.499321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a0675-662d-43f6-aa2d-5e229990c2e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:42:57.499618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a0675-662d-43f6-aa2d-5e229990c2e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:57.501198 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a0675-662d-43f6-aa2d-5e229990c2e9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:57.514887 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 811/1625] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:42:57.520205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9b3db-9fd0-4663-82c3-133ce4052cc8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:57.529444 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9b3db-9fd0-4663-82c3-133ce4052cc8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:57.534790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b0023-9467-4727-8ee3-310dc59cb2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:57.535404 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b0023-9467-4727-8ee3-310dc59cb2ca admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:57.535683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b0023-9467-4727-8ee3-310dc59cb2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:57.535952 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b0023-9467-4727-8ee3-310dc59cb2ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:57.536693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b0023-9467-4727-8ee3-310dc59cb2ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:57.538562 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b0023-9467-4727-8ee3-310dc59cb2ca admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:57.557446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:57.575950 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 815/1626] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:57 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:42:57.576878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a0675-662d-43f6-aa2d-5e229990c2e9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:57.587401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a0675-662d-43f6-aa2d-5e229990c2e9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:57.646096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750107b-721e-4bfb-b16c-61fcdaef2c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:57.646737 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750107b-721e-4bfb-b16c-61fcdaef2c04 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:57.647031 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750107b-721e-4bfb-b16c-61fcdaef2c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:57.647284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750107b-721e-4bfb-b16c-61fcdaef2c04 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:57.647919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5750107b-721e-4bfb-b16c-61fcdaef2c04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:57.649851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5750107b-721e-4bfb-b16c-61fcdaef2c04 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:57.671574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 812/1627] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:57 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:57.675824 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b0023-9467-4727-8ee3-310dc59cb2ca admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:57.685931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b0023-9467-4727-8ee3-310dc59cb2ca admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:57.690573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2562e-a725-457d-b8b8-c22af0e59692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:57.691169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2562e-a725-457d-b8b8-c22af0e59692 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:57.691439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2562e-a725-457d-b8b8-c22af0e59692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:57.691703 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2562e-a725-457d-b8b8-c22af0e59692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:57.692332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2562e-a725-457d-b8b8-c22af0e59692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:57.693908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2562e-a725-457d-b8b8-c22af0e59692 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:57.707376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 816/1628] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:57.708593 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750107b-721e-4bfb-b16c-61fcdaef2c04 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:57.718437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750107b-721e-4bfb-b16c-61fcdaef2c04 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:57.723639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aea4f8-3a02-40ea-bd69-2afe53e35236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:57.724230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aea4f8-3a02-40ea-bd69-2afe53e35236 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:57.724784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aea4f8-3a02-40ea-bd69-2afe53e35236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:57.724784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aea4f8-3a02-40ea-bd69-2afe53e35236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:57.725384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aea4f8-3a02-40ea-bd69-2afe53e35236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:57.727197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aea4f8-3a02-40ea-bd69-2afe53e35236 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:57.739277 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 813/1629] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:57.743496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2562e-a725-457d-b8b8-c22af0e59692 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:57.753684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2562e-a725-457d-b8b8-c22af0e59692 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:57.758280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975e49c-dd3f-4ecc-aa8d-715f68f99efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:57.758920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975e49c-dd3f-4ecc-aa8d-715f68f99efd admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:57.759193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975e49c-dd3f-4ecc-aa8d-715f68f99efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:57.759457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975e49c-dd3f-4ecc-aa8d-715f68f99efd admin admin] PATH_INFO: `/v3/projects/203ba3b1040a4c25971a4b50d9d5fcd9/users/bfc029d743694ea0a57baefb352412c7/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:57.774010 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975e49c-dd3f-4ecc-aa8d-715f68f99efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=203ba3b1040a4c25971a4b50d9d5fcd9, user_id=bfc029d743694ea0a57baefb352412c7, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:57.776643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975e49c-dd3f-4ecc-aa8d-715f68f99efd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:57.790543 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 817/1630] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:57 2022] PUT /identity/v3/projects/203ba3b1040a4c25971a4b50d9d5fcd9/users/bfc029d743694ea0a57baefb352412c7/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:57.801998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7bebb-871a-43eb-93cd-ec0d0a03fbea None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:57.802323 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7bebb-871a-43eb-93cd-ec0d0a03fbea None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:57.802599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7bebb-871a-43eb-93cd-ec0d0a03fbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:57.831065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-08b7bebb-871a-43eb-93cd-ec0d0a03fbea None None] MFA Rules not processed for user `bfc029d743694ea0a57baefb352412c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:57.832173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-08b7bebb-871a-43eb-93cd-ec0d0a03fbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:57.876433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:57.901866 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 814/1631] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:57.906319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975e49c-dd3f-4ecc-aa8d-715f68f99efd admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:57.975466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975e49c-dd3f-4ecc-aa8d-715f68f99efd admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:57.980106 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a92180-5130-43ba-8757-9614f4e910c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:57.980708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a92180-5130-43ba-8757-9614f4e910c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:57.980988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a92180-5130-43ba-8757-9614f4e910c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:57.981259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a92180-5130-43ba-8757-9614f4e910c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:57.981898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a92180-5130-43ba-8757-9614f4e910c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:57.983838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a92180-5130-43ba-8757-9614f4e910c1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:58.009312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 818/1632] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:42:57 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:58.013597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7bebb-871a-43eb-93cd-ec0d0a03fbea None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:58.025233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7bebb-871a-43eb-93cd-ec0d0a03fbea None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:58.081523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b44505-b461-4f67-aab3-f650750f65ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:58.082053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b44505-b461-4f67-aab3-f650750f65ed admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:58.082386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b44505-b461-4f67-aab3-f650750f65ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:58.082659 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b44505-b461-4f67-aab3-f650750f65ed admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:58.083298 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b44505-b461-4f67-aab3-f650750f65ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:58.085224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b44505-b461-4f67-aab3-f650750f65ed admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:58.105769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 815/1633] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:42:58 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:42:58.109792 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a92180-5130-43ba-8757-9614f4e910c1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:58.119961 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a92180-5130-43ba-8757-9614f4e910c1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:58.124708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32068ba-2c65-4336-8d15-9225473b9eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:58.125305 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32068ba-2c65-4336-8d15-9225473b9eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:58.125583 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32068ba-2c65-4336-8d15-9225473b9eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:58.125852 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32068ba-2c65-4336-8d15-9225473b9eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:58.127043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32068ba-2c65-4336-8d15-9225473b9eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:58.128738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32068ba-2c65-4336-8d15-9225473b9eb0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:58.139099 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 819/1634] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:42:58.143881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b44505-b461-4f67-aab3-f650750f65ed admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:58.153692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b44505-b461-4f67-aab3-f650750f65ed admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:58.158493 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2a315-9c3f-4620-ba07-d9d2dd16181c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:58.159219 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2a315-9c3f-4620-ba07-d9d2dd16181c admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:58.159500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2a315-9c3f-4620-ba07-d9d2dd16181c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:58.159774 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2a315-9c3f-4620-ba07-d9d2dd16181c admin admin] PATH_INFO: `/v3/projects/456a320f02dc4e4b898d635031145b4e/users/cd8978a4f8024d5aae12209428602e3b/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:58.174300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2a315-9c3f-4620-ba07-d9d2dd16181c admin admin] RBAC: Authorizing `identity:create_grant(project_id=456a320f02dc4e4b898d635031145b4e, user_id=cd8978a4f8024d5aae12209428602e3b, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:58.176940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2a315-9c3f-4620-ba07-d9d2dd16181c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:58.191239 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 816/1635] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:58 2022] PUT /identity/v3/projects/456a320f02dc4e4b898d635031145b4e/users/cd8978a4f8024d5aae12209428602e3b/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:58.195222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32068ba-2c65-4336-8d15-9225473b9eb0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:58.206779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32068ba-2c65-4336-8d15-9225473b9eb0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:58.270040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea38f9-0e16-48d0-b06a-80a96ceb8f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:58.270944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea38f9-0e16-48d0-b06a-80a96ceb8f2d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:58.271218 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea38f9-0e16-48d0-b06a-80a96ceb8f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:58.271481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea38f9-0e16-48d0-b06a-80a96ceb8f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:58.272762 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea38f9-0e16-48d0-b06a-80a96ceb8f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:58.273912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea38f9-0e16-48d0-b06a-80a96ceb8f2d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:58.285314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 820/1636] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:42:58.289505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2a315-9c3f-4620-ba07-d9d2dd16181c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:58.300365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2a315-9c3f-4620-ba07-d9d2dd16181c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:58.305729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6575ea-ad5e-464e-b1de-3f1a71b3b3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:58.306498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6575ea-ad5e-464e-b1de-3f1a71b3b3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:58.306847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6575ea-ad5e-464e-b1de-3f1a71b3b3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:58.307186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6575ea-ad5e-464e-b1de-3f1a71b3b3a3 admin admin] PATH_INFO: `/v3/projects/456a320f02dc4e4b898d635031145b4e/users/cd8978a4f8024d5aae12209428602e3b/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:58.309469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6575ea-ad5e-464e-b1de-3f1a71b3b3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=456a320f02dc4e4b898d635031145b4e, user_id=cd8978a4f8024d5aae12209428602e3b, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:58.312195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6575ea-ad5e-464e-b1de-3f1a71b3b3a3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:58.324932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 817/1637] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:42:58 2022] PUT /identity/v3/projects/456a320f02dc4e4b898d635031145b4e/users/cd8978a4f8024d5aae12209428602e3b/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:42:58.336988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4991-9ce1-472e-bc1a-549addb38ada None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:58.337381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4991-9ce1-472e-bc1a-549addb38ada None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:58.337661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4991-9ce1-472e-bc1a-549addb38ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:58.369179 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e13c4991-9ce1-472e-bc1a-549addb38ada None None] MFA Rules not processed for user `cd8978a4f8024d5aae12209428602e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:58.370431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e13c4991-9ce1-472e-bc1a-549addb38ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:58.388756 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:58.443430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 821/1638] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:58.454194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6575ea-ad5e-464e-b1de-3f1a71b3b3a3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:58.456075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7a6575ea-ad5e-464e-b1de-3f1a71b3b3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:58.512628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6575ea-ad5e-464e-b1de-3f1a71b3b3a3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:58.516220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e77c13-3953-42ce-9e33-283dcdfa2b32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:58.516825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e77c13-3953-42ce-9e33-283dcdfa2b32 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:58.517106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e77c13-3953-42ce-9e33-283dcdfa2b32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:58.517458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e77c13-3953-42ce-9e33-283dcdfa2b32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:58.526565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e77c13-3953-42ce-9e33-283dcdfa2b32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:58.527979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e77c13-3953-42ce-9e33-283dcdfa2b32 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:58.537643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 818/1639] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:59.207611 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4991-9ce1-472e-bc1a-549addb38ada None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:59.273995 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4991-9ce1-472e-bc1a-549addb38ada None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:59.278556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:59.279189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:59.279468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:59.279738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] PATH_INFO: `/v3/users/bfc029d743694ea0a57baefb352412c7` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:59.280852 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc029d743694ea0a57baefb352412c7)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:59.283691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:59.329199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc029d743694ea0a57baefb352412c7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:59.330437 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc029d743694ea0a57baefb352412c7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:59.342122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc029d743694ea0a57baefb352412c7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:59.342780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc029d743694ea0a57baefb352412c7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:59.350543 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc029d743694ea0a57baefb352412c7'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:59.363103 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 822/1640] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:59 2022] DELETE /identity/v3/users/bfc029d743694ea0a57baefb352412c7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:59.367492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a99d8b-153c-4b54-ba1a-11cb2cf4fd9d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:59.367979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a99d8b-153c-4b54-ba1a-11cb2cf4fd9d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:59.368071 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a99d8b-153c-4b54-ba1a-11cb2cf4fd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:59.387164 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-19a99d8b-153c-4b54-ba1a-11cb2cf4fd9d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:42:59.388467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-19a99d8b-153c-4b54-ba1a-11cb2cf4fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:59.429793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:59.456841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 819/1641] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:42:59.466873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:59.468269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:59.511256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:59.534135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7ea2-9bc1-4858-8514-5458fd71bb9d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:59.538140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee48ca-3aa7-43fc-8b2b-f7e542f08f0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:59.538778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee48ca-3aa7-43fc-8b2b-f7e542f08f0a service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:59.539050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee48ca-3aa7-43fc-8b2b-f7e542f08f0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:59.539314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee48ca-3aa7-43fc-8b2b-f7e542f08f0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:59.551420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee48ca-3aa7-43fc-8b2b-f7e542f08f0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:59.552928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee48ca-3aa7-43fc-8b2b-f7e542f08f0a service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:59.562965 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 823/1642] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:59.842077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a99d8b-153c-4b54-ba1a-11cb2cf4fd9d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:59.844147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-19a99d8b-153c-4b54-ba1a-11cb2cf4fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:59.860080 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a99d8b-153c-4b54-ba1a-11cb2cf4fd9d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:59.875208 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:59.875955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:59.876314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:59.877204 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] PATH_INFO: `/v3/projects/203ba3b1040a4c25971a4b50d9d5fcd9` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:59.878438 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=203ba3b1040a4c25971a4b50d9d5fcd9)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:59.880418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:59.940952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203ba3b1040a4c25971a4b50d9d5fcd9', 'request_id': 'req-3e45309d-b297-4652-9a57-aaba7d1818c2'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:59.950332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203ba3b1040a4c25971a4b50d9d5fcd9', 'request_id': 'req-3e45309d-b297-4652-9a57-aaba7d1818c2'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:59.958020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203ba3b1040a4c25971a4b50d9d5fcd9', 'request_id': 'req-3e45309d-b297-4652-9a57-aaba7d1818c2'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:59.969780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] The token cache is being invalidate because project 203ba3b1040a4c25971a4b50d9d5fcd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:59.970072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:59.970835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:59.972682 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 820/1643] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:59 2022] DELETE /identity/v3/projects/203ba3b1040a4c25971a4b50d9d5fcd9 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:59.976735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee48ca-3aa7-43fc-8b2b-f7e542f08f0a service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:59.989253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee48ca-3aa7-43fc-8b2b-f7e542f08f0a service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:00.052323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235e7cc-6f5c-4caf-a67a-28062f38c778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:00.052962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235e7cc-6f5c-4caf-a67a-28062f38c778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:00.053244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235e7cc-6f5c-4caf-a67a-28062f38c778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:00.053514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235e7cc-6f5c-4caf-a67a-28062f38c778 admin admin] PATH_INFO: `/v3/users/cd8978a4f8024d5aae12209428602e3b` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:00.054878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235e7cc-6f5c-4caf-a67a-28062f38c778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd8978a4f8024d5aae12209428602e3b)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:00.056978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235e7cc-6f5c-4caf-a67a-28062f38c778 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:00.107805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a235e7cc-6f5c-4caf-a67a-28062f38c778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd8978a4f8024d5aae12209428602e3b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:00.108490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a235e7cc-6f5c-4caf-a67a-28062f38c778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd8978a4f8024d5aae12209428602e3b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:00.121560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a235e7cc-6f5c-4caf-a67a-28062f38c778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd8978a4f8024d5aae12209428602e3b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:00.122192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a235e7cc-6f5c-4caf-a67a-28062f38c778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd8978a4f8024d5aae12209428602e3b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:00.129438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a235e7cc-6f5c-4caf-a67a-28062f38c778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd8978a4f8024d5aae12209428602e3b'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:00.145272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 824/1644] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:59 2022] DELETE /identity/v3/users/cd8978a4f8024d5aae12209428602e3b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:00.431163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:00.432298 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:00.452662 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45309d-b297-4652-9a57-aaba7d1818c2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:00.511560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:00.512192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:00.512474 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:00.512747 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] PATH_INFO: `/v3/projects/456a320f02dc4e4b898d635031145b4e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:00.513861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=456a320f02dc4e4b898d635031145b4e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:00.515792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:00.561734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '456a320f02dc4e4b898d635031145b4e', 'request_id': 'req-59a34d4e-158f-497e-ae92-86525c317b97'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:00.570408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '456a320f02dc4e4b898d635031145b4e', 'request_id': 'req-59a34d4e-158f-497e-ae92-86525c317b97'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:00.578288 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '456a320f02dc4e4b898d635031145b4e', 'request_id': 'req-59a34d4e-158f-497e-ae92-86525c317b97'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:00.582950 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:00.591540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] The token cache is being invalidate because project 456a320f02dc4e4b898d635031145b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:00.592120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:00.593002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-59a34d4e-158f-497e-ae92-86525c317b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:00.594969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 821/1645] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:00 2022] DELETE /identity/v3/projects/456a320f02dc4e4b898d635031145b4e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:00.599871 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa81ea-5e1b-40eb-bd0c-39f4e2f4d73b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:00.600346 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa81ea-5e1b-40eb-bd0c-39f4e2f4d73b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:00.600437 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa81ea-5e1b-40eb-bd0c-39f4e2f4d73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:00.621089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-28fa81ea-5e1b-40eb-bd0c-39f4e2f4d73b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:00.622478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-28fa81ea-5e1b-40eb-bd0c-39f4e2f4d73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:00.686043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:00.705350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 825/1646] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:00.706770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59234a-7756-4e4f-babc-f52266c501d2 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:00.707262 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59234a-7756-4e4f-babc-f52266c501d2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:00.707622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59234a-7756-4e4f-babc-f52266c501d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:00.730078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2a59234a-7756-4e4f-babc-f52266c501d2 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:00.731975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2a59234a-7756-4e4f-babc-f52266c501d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:00.742973 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 822/1647] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:43:00.747349 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa81ea-5e1b-40eb-bd0c-39f4e2f4d73b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:00.766315 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa81ea-5e1b-40eb-bd0c-39f4e2f4d73b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:00.771563 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ebc98-f4ba-4271-93d7-a12bbcd07eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:00.771891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ebc98-f4ba-4271-93d7-a12bbcd07eda admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:00.772174 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ebc98-f4ba-4271-93d7-a12bbcd07eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:00.772447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ebc98-f4ba-4271-93d7-a12bbcd07eda admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:00.773776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ebc98-f4ba-4271-93d7-a12bbcd07eda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:00.773776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ebc98-f4ba-4271-93d7-a12bbcd07eda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:00.775406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ebc98-f4ba-4271-93d7-a12bbcd07eda admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:00.790229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 826/1648] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:00.794451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59234a-7756-4e4f-babc-f52266c501d2 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:00.804656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59234a-7756-4e4f-babc-f52266c501d2 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:00.809490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b70c38-2d02-4768-a8c2-e46f4e93f9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:00.810094 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b70c38-2d02-4768-a8c2-e46f4e93f9ab admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:00.810403 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b70c38-2d02-4768-a8c2-e46f4e93f9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:00.810680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b70c38-2d02-4768-a8c2-e46f4e93f9ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:00.811324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b70c38-2d02-4768-a8c2-e46f4e93f9ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:00.813221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b70c38-2d02-4768-a8c2-e46f4e93f9ab admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:00.840320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 823/1649] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:00 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:43:00.844427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ebc98-f4ba-4271-93d7-a12bbcd07eda admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:00.854774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ebc98-f4ba-4271-93d7-a12bbcd07eda admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:00.911548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b9844-90f2-4ae0-9880-30ff93cd312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:00.912080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b9844-90f2-4ae0-9880-30ff93cd312a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:00.912369 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b9844-90f2-4ae0-9880-30ff93cd312a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:00.912632 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b9844-90f2-4ae0-9880-30ff93cd312a admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:00.913266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b9844-90f2-4ae0-9880-30ff93cd312a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:00.915234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b9844-90f2-4ae0-9880-30ff93cd312a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:00.935887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 827/1650] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:00 2022] POST /identity/v3/users => generated 280 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:00.940295 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b70c38-2d02-4768-a8c2-e46f4e93f9ab admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:00.950502 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b70c38-2d02-4768-a8c2-e46f4e93f9ab admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:00.955638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e958f8-29e1-4526-a41e-6a3d1c50cf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:00.956230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e958f8-29e1-4526-a41e-6a3d1c50cf46 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:00.956511 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e958f8-29e1-4526-a41e-6a3d1c50cf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:00.956781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e958f8-29e1-4526-a41e-6a3d1c50cf46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:00.957418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e958f8-29e1-4526-a41e-6a3d1c50cf46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:00.959040 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e958f8-29e1-4526-a41e-6a3d1c50cf46 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:00.968863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 824/1651] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:00.973369 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b9844-90f2-4ae0-9880-30ff93cd312a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:00.983438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b9844-90f2-4ae0-9880-30ff93cd312a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:00.988196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3e049-a31e-47bb-be46-ecc812224be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:00.988816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3e049-a31e-47bb-be46-ecc812224be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:00.989095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3e049-a31e-47bb-be46-ecc812224be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:00.989370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3e049-a31e-47bb-be46-ecc812224be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:00.990037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3e049-a31e-47bb-be46-ecc812224be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:00.991659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3e049-a31e-47bb-be46-ecc812224be0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:01.002275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 828/1652] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:01.006474 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e958f8-29e1-4526-a41e-6a3d1c50cf46 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:01.016439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e958f8-29e1-4526-a41e-6a3d1c50cf46 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:01.021573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240efae-bbeb-44bf-80c1-8666b8561fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:01.022277 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240efae-bbeb-44bf-80c1-8666b8561fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:01.022554 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240efae-bbeb-44bf-80c1-8666b8561fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:01.022822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240efae-bbeb-44bf-80c1-8666b8561fb9 admin admin] PATH_INFO: `/v3/projects/a03525a1ae5346b8a96e572660e75d1c/users/4ba2a644f85f4099a1a7c112460690e8/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:01.038294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240efae-bbeb-44bf-80c1-8666b8561fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a03525a1ae5346b8a96e572660e75d1c, user_id=4ba2a644f85f4099a1a7c112460690e8, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:01.041245 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240efae-bbeb-44bf-80c1-8666b8561fb9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:01.054564 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 825/1653] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:01 2022] PUT /identity/v3/projects/a03525a1ae5346b8a96e572660e75d1c/users/4ba2a644f85f4099a1a7c112460690e8/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:01.066276 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c3a2c-1db7-4b68-ad53-c885806e01da None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:01.066695 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c3a2c-1db7-4b68-ad53-c885806e01da None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:01.066971 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c3a2c-1db7-4b68-ad53-c885806e01da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:01.095506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-da1c3a2c-1db7-4b68-ad53-c885806e01da None None] MFA Rules not processed for user `4ba2a644f85f4099a1a7c112460690e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:01.096906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-da1c3a2c-1db7-4b68-ad53-c885806e01da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:01.168468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 829/1654] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:01.215911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240efae-bbeb-44bf-80c1-8666b8561fb9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:01.218261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2240efae-bbeb-44bf-80c1-8666b8561fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:01.282772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240efae-bbeb-44bf-80c1-8666b8561fb9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:01.286779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375e9d2-2273-405e-bb0f-167ccbaeb656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:01.287395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375e9d2-2273-405e-bb0f-167ccbaeb656 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:01.287677 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375e9d2-2273-405e-bb0f-167ccbaeb656 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:01.287952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375e9d2-2273-405e-bb0f-167ccbaeb656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:01.296961 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375e9d2-2273-405e-bb0f-167ccbaeb656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:01.298471 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375e9d2-2273-405e-bb0f-167ccbaeb656 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:01.308954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 826/1655] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:01 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:01.325373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c3a2c-1db7-4b68-ad53-c885806e01da None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:01.378465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:01.390454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c3a2c-1db7-4b68-ad53-c885806e01da None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:01.395999 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:01.396626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:01.396902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:01.397167 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] PATH_INFO: `/v3/users/4ba2a644f85f4099a1a7c112460690e8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:01.398310 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ba2a644f85f4099a1a7c112460690e8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:01.400194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:01.447231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ba2a644f85f4099a1a7c112460690e8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:01.448979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ba2a644f85f4099a1a7c112460690e8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:01.462568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ba2a644f85f4099a1a7c112460690e8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:01.463629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ba2a644f85f4099a1a7c112460690e8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:01.475113 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ba2a644f85f4099a1a7c112460690e8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:01.494496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 830/1656] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:01 2022] DELETE /identity/v3/users/4ba2a644f85f4099a1a7c112460690e8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:01.500231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec5ef0-00eb-4686-8150-7c61328d2b54 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:01.500954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec5ef0-00eb-4686-8150-7c61328d2b54 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:01.500954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec5ef0-00eb-4686-8150-7c61328d2b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:01.520221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4aec5ef0-00eb-4686-8150-7c61328d2b54 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:01.521405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4aec5ef0-00eb-4686-8150-7c61328d2b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:01.581839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 827/1657] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:43:01.593464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:01.594997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:01.654371 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704a016-7c7b-4a7e-af1e-ba5ebd791612 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:01.658944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2bf20-9a32-45df-9827-290076fee246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:01.659554 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2bf20-9a32-45df-9827-290076fee246 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:01.659835 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2bf20-9a32-45df-9827-290076fee246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:01.660107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2bf20-9a32-45df-9827-290076fee246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:01.668904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2bf20-9a32-45df-9827-290076fee246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:01.670352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2bf20-9a32-45df-9827-290076fee246 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:01.679794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 831/1658] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:02.049786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec5ef0-00eb-4686-8150-7c61328d2b54 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:02.051320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4aec5ef0-00eb-4686-8150-7c61328d2b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:02.067722 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec5ef0-00eb-4686-8150-7c61328d2b54 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:02.072282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:02.073620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:02.074339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:02.074608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] PATH_INFO: `/v3/projects/a03525a1ae5346b8a96e572660e75d1c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:02.075724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a03525a1ae5346b8a96e572660e75d1c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:02.077688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:02.109808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:02.128035 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a03525a1ae5346b8a96e572660e75d1c', 'request_id': 'req-85ccc9af-8cd0-407d-811a-5023df26e3d6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:02.134871 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a03525a1ae5346b8a96e572660e75d1c', 'request_id': 'req-85ccc9af-8cd0-407d-811a-5023df26e3d6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:02.141019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a03525a1ae5346b8a96e572660e75d1c', 'request_id': 'req-85ccc9af-8cd0-407d-811a-5023df26e3d6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:02.152058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] The token cache is being invalidate because project a03525a1ae5346b8a96e572660e75d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:02.152342 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:02.152966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-85ccc9af-8cd0-407d-811a-5023df26e3d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:02.154500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 828/1659] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:02 2022] DELETE /identity/v3/projects/a03525a1ae5346b8a96e572660e75d1c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:02.160610 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de2d91-66e9-4e19-9719-2b0c21e7357c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:02.160893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de2d91-66e9-4e19-9719-2b0c21e7357c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:02.161164 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de2d91-66e9-4e19-9719-2b0c21e7357c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:02.181069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-25de2d91-66e9-4e19-9719-2b0c21e7357c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:02.182303 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-25de2d91-66e9-4e19-9719-2b0c21e7357c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:02.240547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 832/1660] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:02.249240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e29cb-3a09-4169-854b-163e4bb95cee None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:02.249532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e29cb-3a09-4169-854b-163e4bb95cee None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:02.249802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e29cb-3a09-4169-854b-163e4bb95cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:02.270359 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-968e29cb-3a09-4169-854b-163e4bb95cee None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:02.271835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-968e29cb-3a09-4169-854b-163e4bb95cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:02.280135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 829/1661] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:02.285798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de2d91-66e9-4e19-9719-2b0c21e7357c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:02.295555 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de2d91-66e9-4e19-9719-2b0c21e7357c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:02.302390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe4c58-2216-474c-81c8-aa9a312048ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:02.303020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe4c58-2216-474c-81c8-aa9a312048ad admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:02.303287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe4c58-2216-474c-81c8-aa9a312048ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:02.303554 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe4c58-2216-474c-81c8-aa9a312048ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:02.304203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe4c58-2216-474c-81c8-aa9a312048ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:02.304504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe4c58-2216-474c-81c8-aa9a312048ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:02.306731 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe4c58-2216-474c-81c8-aa9a312048ad admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:02.322002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 833/1662] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:02.326137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e29cb-3a09-4169-854b-163e4bb95cee None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:02.336088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e29cb-3a09-4169-854b-163e4bb95cee None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:02.340861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74c076-ee54-446f-adc9-c1aede56a36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:02.341458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74c076-ee54-446f-adc9-c1aede56a36f admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:02.341734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74c076-ee54-446f-adc9-c1aede56a36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:02.341998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74c076-ee54-446f-adc9-c1aede56a36f admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:02.342667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74c076-ee54-446f-adc9-c1aede56a36f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:02.344546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74c076-ee54-446f-adc9-c1aede56a36f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:02.372842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 830/1663] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:02 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:43:02.377806 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe4c58-2216-474c-81c8-aa9a312048ad admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:02.382958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:02.389873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe4c58-2216-474c-81c8-aa9a312048ad admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:02.447218 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492d4f7-52e0-45f0-8c45-42cfd318ed4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:02.447827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492d4f7-52e0-45f0-8c45-42cfd318ed4b admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:02.448108 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492d4f7-52e0-45f0-8c45-42cfd318ed4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:02.448380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492d4f7-52e0-45f0-8c45-42cfd318ed4b admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:02.449014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492d4f7-52e0-45f0-8c45-42cfd318ed4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:02.450983 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492d4f7-52e0-45f0-8c45-42cfd318ed4b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:02.471674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 834/1664] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:02 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:43:02.475674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74c076-ee54-446f-adc9-c1aede56a36f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:02.486477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74c076-ee54-446f-adc9-c1aede56a36f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:02.491164 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b10f3-6f18-4f43-a6b8-acb8928f511e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:02.491756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b10f3-6f18-4f43-a6b8-acb8928f511e admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:02.492094 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b10f3-6f18-4f43-a6b8-acb8928f511e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:02.498010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b10f3-6f18-4f43-a6b8-acb8928f511e admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:02.498010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b10f3-6f18-4f43-a6b8-acb8928f511e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:02.498010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b10f3-6f18-4f43-a6b8-acb8928f511e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:02.505528 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 831/1665] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:02.509431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492d4f7-52e0-45f0-8c45-42cfd318ed4b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:02.519778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492d4f7-52e0-45f0-8c45-42cfd318ed4b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:02.524336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e523280-d3a1-4193-b0b3-a122d64c3685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:02.524923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e523280-d3a1-4193-b0b3-a122d64c3685 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:02.525226 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e523280-d3a1-4193-b0b3-a122d64c3685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:02.525526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e523280-d3a1-4193-b0b3-a122d64c3685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:02.526157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e523280-d3a1-4193-b0b3-a122d64c3685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:02.527747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e523280-d3a1-4193-b0b3-a122d64c3685 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:02.539171 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 835/1666] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:02.543242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b10f3-6f18-4f43-a6b8-acb8928f511e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:02.553624 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b10f3-6f18-4f43-a6b8-acb8928f511e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:02.558125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71202e6b-0ab5-487b-a1a4-c4107f37e35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:02.558781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71202e6b-0ab5-487b-a1a4-c4107f37e35b admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:02.559060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71202e6b-0ab5-487b-a1a4-c4107f37e35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:02.559331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71202e6b-0ab5-487b-a1a4-c4107f37e35b admin admin] PATH_INFO: `/v3/projects/708306fc9c2f43b98575500ad89bdf48/users/8703990627cd456da9f2075fa07bb090/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:02.573422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71202e6b-0ab5-487b-a1a4-c4107f37e35b admin admin] RBAC: Authorizing `identity:create_grant(project_id=708306fc9c2f43b98575500ad89bdf48, user_id=8703990627cd456da9f2075fa07bb090, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:02.576077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71202e6b-0ab5-487b-a1a4-c4107f37e35b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:02.589573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 832/1667] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:02 2022] PUT /identity/v3/projects/708306fc9c2f43b98575500ad89bdf48/users/8703990627cd456da9f2075fa07bb090/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:02.600445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d7ba0-23b9-4f46-b8bf-d6393e14e520 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:02.600730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d7ba0-23b9-4f46-b8bf-d6393e14e520 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:02.601003 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d7ba0-23b9-4f46-b8bf-d6393e14e520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:02.629566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f85d7ba0-23b9-4f46-b8bf-d6393e14e520 None None] MFA Rules not processed for user `8703990627cd456da9f2075fa07bb090`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:02.630779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f85d7ba0-23b9-4f46-b8bf-d6393e14e520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:02.696306 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 836/1668] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:02.708011 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71202e6b-0ab5-487b-a1a4-c4107f37e35b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:02.709707 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-71202e6b-0ab5-487b-a1a4-c4107f37e35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:02.771181 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71202e6b-0ab5-487b-a1a4-c4107f37e35b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:02.774743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c6257-0342-470d-a4db-108902f436a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:02.775352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c6257-0342-470d-a4db-108902f436a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:02.775632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c6257-0342-470d-a4db-108902f436a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:02.775904 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c6257-0342-470d-a4db-108902f436a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:02.785150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c6257-0342-470d-a4db-108902f436a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:02.786620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c6257-0342-470d-a4db-108902f436a6 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:02.800271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 833/1669] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:07.592556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d7ba0-23b9-4f46-b8bf-d6393e14e520 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:07.655199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d7ba0-23b9-4f46-b8bf-d6393e14e520 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:07.660839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:07.661489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:07.661769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:07.662053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] PATH_INFO: `/v3/users/8703990627cd456da9f2075fa07bb090` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:07.663211 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8703990627cd456da9f2075fa07bb090)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:07.665122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:07.678434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:07.714511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8703990627cd456da9f2075fa07bb090'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:07.715169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8703990627cd456da9f2075fa07bb090'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:07.727808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8703990627cd456da9f2075fa07bb090'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:07.728499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8703990627cd456da9f2075fa07bb090'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:07.735265 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8703990627cd456da9f2075fa07bb090'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:07.754383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 837/1670] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:07 2022] DELETE /identity/v3/users/8703990627cd456da9f2075fa07bb090 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:07.756130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463888-553f-48a4-8f0c-5034b5634080 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:07.756622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463888-553f-48a4-8f0c-5034b5634080 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:07.756717 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463888-553f-48a4-8f0c-5034b5634080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:07.777324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f9463888-553f-48a4-8f0c-5034b5634080 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:07.778304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f9463888-553f-48a4-8f0c-5034b5634080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:07.835565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:07.854046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 834/1671] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:07.885149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:07.888129 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:07.966810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bec4f5-26db-4266-b456-3d7022be5265 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:07.975201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5c43a-260d-4c52-8eee-7e6f6679c3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:07.976489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5c43a-260d-4c52-8eee-7e6f6679c3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:07.976967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5c43a-260d-4c52-8eee-7e6f6679c3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:07.977419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5c43a-260d-4c52-8eee-7e6f6679c3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:07.995769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5c43a-260d-4c52-8eee-7e6f6679c3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:07.997815 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5c43a-260d-4c52-8eee-7e6f6679c3c0 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:08.012918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 838/1672] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:08.261680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9463888-553f-48a4-8f0c-5034b5634080 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:08.263144 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f9463888-553f-48a4-8f0c-5034b5634080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:08.417459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9463888-553f-48a4-8f0c-5034b5634080 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:08.432022 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42839720-30ea-4d2c-a308-5e2ac82d3f5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:08.432906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42839720-30ea-4d2c-a308-5e2ac82d3f5f service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:08.433332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42839720-30ea-4d2c-a308-5e2ac82d3f5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:08.433774 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42839720-30ea-4d2c-a308-5e2ac82d3f5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:08.452058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-42839720-30ea-4d2c-a308-5e2ac82d3f5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:08.507334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5c43a-260d-4c52-8eee-7e6f6679c3c0 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:08.510450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0eb5c43a-260d-4c52-8eee-7e6f6679c3c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:08.535831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5c43a-260d-4c52-8eee-7e6f6679c3c0 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:08.543747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:08.544426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:08.544713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:08.544987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] PATH_INFO: `/v3/users/fa9c44dd5b1548d5adc2677b0b46aaab` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:08.546359 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9c44dd5b1548d5adc2677b0b46aaab)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:08.548898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:08.599097 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42839720-30ea-4d2c-a308-5e2ac82d3f5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:08.600813 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42839720-30ea-4d2c-a308-5e2ac82d3f5f service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:08.611158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9c44dd5b1548d5adc2677b0b46aaab'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:08.613300 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9c44dd5b1548d5adc2677b0b46aaab'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:08.614657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 835/1673] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:08.631048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9c44dd5b1548d5adc2677b0b46aaab'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:08.632302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9c44dd5b1548d5adc2677b0b46aaab'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:08.641021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9c44dd5b1548d5adc2677b0b46aaab'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:08.662272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 839/1674] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:08 2022] DELETE /identity/v3/users/fa9c44dd5b1548d5adc2677b0b46aaab => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:08.667018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d2918-fc90-46de-9b9c-0096796034f1 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:08.667311 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d2918-fc90-46de-9b9c-0096796034f1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:08.667695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d2918-fc90-46de-9b9c-0096796034f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:08.690989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-172d2918-fc90-46de-9b9c-0096796034f1 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:08.692600 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-172d2918-fc90-46de-9b9c-0096796034f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:08.788306 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 836/1675] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:08.869997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:08.871530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:08.882204 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:08.970204 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7568b-ae96-41f0-9f51-c68c39e01ff9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:08.979144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce27d7-276e-40d7-9d16-43316c94edba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:08.979974 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce27d7-276e-40d7-9d16-43316c94edba service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:08.980490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce27d7-276e-40d7-9d16-43316c94edba service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:08.980824 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce27d7-276e-40d7-9d16-43316c94edba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:08.993833 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce27d7-276e-40d7-9d16-43316c94edba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:08.995360 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce27d7-276e-40d7-9d16-43316c94edba service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:09.008309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 840/1676] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:09.274440 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d2918-fc90-46de-9b9c-0096796034f1 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:09.276172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-172d2918-fc90-46de-9b9c-0096796034f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:09.293975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d2918-fc90-46de-9b9c-0096796034f1 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:09.299379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:09.300000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:09.300441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:09.300696 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] PATH_INFO: `/v3/projects/708306fc9c2f43b98575500ad89bdf48` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:09.301830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=708306fc9c2f43b98575500ad89bdf48)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:09.303943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:09.367247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:09.379795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '708306fc9c2f43b98575500ad89bdf48', 'request_id': 'req-9175d1e2-91a3-4ae4-81e7-0fae531fb080'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:09.411021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '708306fc9c2f43b98575500ad89bdf48', 'request_id': 'req-9175d1e2-91a3-4ae4-81e7-0fae531fb080'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:09.424079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '708306fc9c2f43b98575500ad89bdf48', 'request_id': 'req-9175d1e2-91a3-4ae4-81e7-0fae531fb080'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:09.447277 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] The token cache is being invalidate because project 708306fc9c2f43b98575500ad89bdf48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:09.448215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:09.449290 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9175d1e2-91a3-4ae4-81e7-0fae531fb080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:09.451443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 837/1677] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:09 2022] DELETE /identity/v3/projects/708306fc9c2f43b98575500ad89bdf48 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:09.457680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9da0c-90b4-4742-9cd2-a7fe5acf6b1f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:09.457968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9da0c-90b4-4742-9cd2-a7fe5acf6b1f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:09.458295 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9da0c-90b4-4742-9cd2-a7fe5acf6b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:09.496786 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-89b9da0c-90b4-4742-9cd2-a7fe5acf6b1f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:09.498737 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-89b9da0c-90b4-4742-9cd2-a7fe5acf6b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:09.584185 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 841/1678] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:43:09.592062 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f22d9-6149-44ff-b9b2-320432cbb23a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:09.592639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f22d9-6149-44ff-b9b2-320432cbb23a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:09.593259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f22d9-6149-44ff-b9b2-320432cbb23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:09.618275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b08f22d9-6149-44ff-b9b2-320432cbb23a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:09.619896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b08f22d9-6149-44ff-b9b2-320432cbb23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:09.631039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 838/1679] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:43:09.634659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9da0c-90b4-4742-9cd2-a7fe5acf6b1f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:09.647506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9da0c-90b4-4742-9cd2-a7fe5acf6b1f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:09.654344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25943634-af26-4867-935a-29ce45967921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:09.655251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25943634-af26-4867-935a-29ce45967921 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:09.655780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25943634-af26-4867-935a-29ce45967921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:09.656247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25943634-af26-4867-935a-29ce45967921 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:09.657119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25943634-af26-4867-935a-29ce45967921 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:09.658691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25943634-af26-4867-935a-29ce45967921 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:09.660870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25943634-af26-4867-935a-29ce45967921 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:09.688160 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 842/1680] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:09.693534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f22d9-6149-44ff-b9b2-320432cbb23a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:09.703692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f22d9-6149-44ff-b9b2-320432cbb23a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:09.709751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021eac54-139b-4735-94e2-274d942b3319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:09.710650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021eac54-139b-4735-94e2-274d942b3319 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:09.711132 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021eac54-139b-4735-94e2-274d942b3319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:09.711601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021eac54-139b-4735-94e2-274d942b3319 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:09.712462 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021eac54-139b-4735-94e2-274d942b3319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:09.714903 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021eac54-139b-4735-94e2-274d942b3319 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:09.751333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 839/1681] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:09 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:09.756241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25943634-af26-4867-935a-29ce45967921 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:09.769716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25943634-af26-4867-935a-29ce45967921 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:09.852163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaba30-2845-4854-9b90-fde83f9858e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:09.853613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaba30-2845-4854-9b90-fde83f9858e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:09.854017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaba30-2845-4854-9b90-fde83f9858e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:09.854435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaaba30-2845-4854-9b90-fde83f9858e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:09.855156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaba30-2845-4854-9b90-fde83f9858e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:09.857229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaaba30-2845-4854-9b90-fde83f9858e4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:09.886092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 843/1682] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:09 2022] POST /identity/v3/users => generated 283 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:09.891212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021eac54-139b-4735-94e2-274d942b3319 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:09.912134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021eac54-139b-4735-94e2-274d942b3319 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:09.920870 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748ff6a-99d9-4d1b-b098-75886ca839d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:09.921519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748ff6a-99d9-4d1b-b098-75886ca839d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:09.921899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748ff6a-99d9-4d1b-b098-75886ca839d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:09.922320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748ff6a-99d9-4d1b-b098-75886ca839d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:09.923587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748ff6a-99d9-4d1b-b098-75886ca839d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:09.926231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748ff6a-99d9-4d1b-b098-75886ca839d7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:09.939387 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 840/1683] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:09.943447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaba30-2845-4854-9b90-fde83f9858e4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:09.953901 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaaba30-2845-4854-9b90-fde83f9858e4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:09.961352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb9745-d227-45c6-82ee-519c800c4458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:09.962038 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb9745-d227-45c6-82ee-519c800c4458 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:09.962365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb9745-d227-45c6-82ee-519c800c4458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:09.962740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb9745-d227-45c6-82ee-519c800c4458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:09.963443 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb9745-d227-45c6-82ee-519c800c4458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:09.965534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb9745-d227-45c6-82ee-519c800c4458 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:09.987089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 844/1684] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:09.991128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748ff6a-99d9-4d1b-b098-75886ca839d7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:10.005207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748ff6a-99d9-4d1b-b098-75886ca839d7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:10.010883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e761b-b293-40b2-9e2e-8152f9f2d9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:10.012042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e761b-b293-40b2-9e2e-8152f9f2d9e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:10.012436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e761b-b293-40b2-9e2e-8152f9f2d9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:10.012815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e761b-b293-40b2-9e2e-8152f9f2d9e3 admin admin] PATH_INFO: `/v3/projects/c0c63e842bab45b3aaaf36257175c04c/users/bb3858c54a4842239150d8645bdafbf2/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:10.031316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e761b-b293-40b2-9e2e-8152f9f2d9e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0c63e842bab45b3aaaf36257175c04c, user_id=bb3858c54a4842239150d8645bdafbf2, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:10.035733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e761b-b293-40b2-9e2e-8152f9f2d9e3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:10.058527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 841/1685] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:09 2022] PUT /identity/v3/projects/c0c63e842bab45b3aaaf36257175c04c/users/bb3858c54a4842239150d8645bdafbf2/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:10.072422 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535679b9-06b0-4a4a-8eb6-9efe738b8177 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:10.072707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535679b9-06b0-4a4a-8eb6-9efe738b8177 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:10.073000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535679b9-06b0-4a4a-8eb6-9efe738b8177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:10.141033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:10.155657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-535679b9-06b0-4a4a-8eb6-9efe738b8177 None None] MFA Rules not processed for user `bb3858c54a4842239150d8645bdafbf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:10.157975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-535679b9-06b0-4a4a-8eb6-9efe738b8177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:10.293551 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 845/1686] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:10.315526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e761b-b293-40b2-9e2e-8152f9f2d9e3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:10.320622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bf2e761b-b293-40b2-9e2e-8152f9f2d9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:10.442867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e761b-b293-40b2-9e2e-8152f9f2d9e3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:10.447646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324420c-55b5-4535-b9be-2de64ca24000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:10.448279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324420c-55b5-4535-b9be-2de64ca24000 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:10.448593 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324420c-55b5-4535-b9be-2de64ca24000 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:10.448871 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324420c-55b5-4535-b9be-2de64ca24000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:10.468696 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324420c-55b5-4535-b9be-2de64ca24000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:10.470699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324420c-55b5-4535-b9be-2de64ca24000 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:10.487171 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 842/1687] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:10.609964 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535679b9-06b0-4a4a-8eb6-9efe738b8177 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:10.612116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-535679b9-06b0-4a4a-8eb6-9efe738b8177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:10.817201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535679b9-06b0-4a4a-8eb6-9efe738b8177 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:10.825369 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:10.826377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:10.827346 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:10.827735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] PATH_INFO: `/v3/projects/45bebd4d70fe4613a0c21995128615c3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:10.829204 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45bebd4d70fe4613a0c21995128615c3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:10.833186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:10.875880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324420c-55b5-4535-b9be-2de64ca24000 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:10.878101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c324420c-55b5-4535-b9be-2de64ca24000 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:10.906200 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324420c-55b5-4535-b9be-2de64ca24000 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:10.910742 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45bebd4d70fe4613a0c21995128615c3', 'request_id': 'req-0215be2a-6d61-4949-9201-fc013f45d0f3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:10.919905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45bebd4d70fe4613a0c21995128615c3', 'request_id': 'req-0215be2a-6d61-4949-9201-fc013f45d0f3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:10.933964 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45bebd4d70fe4613a0c21995128615c3', 'request_id': 'req-0215be2a-6d61-4949-9201-fc013f45d0f3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:10.957225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] The token cache is being invalidate because project 45bebd4d70fe4613a0c21995128615c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:10.957867 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:10.958831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:10.965561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 846/1688] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:10 2022] DELETE /identity/v3/projects/45bebd4d70fe4613a0c21995128615c3 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:10.968060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:10.970942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:11.104472 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bdbc2-6d22-46b2-a761-839a636062bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:11.105179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bdbc2-6d22-46b2-a761-839a636062bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:11.105629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bdbc2-6d22-46b2-a761-839a636062bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:11.106045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bdbc2-6d22-46b2-a761-839a636062bb admin admin] PATH_INFO: `/v3/users/6501053893ad408db9e647e5ce3b2afe` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:11.108784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bdbc2-6d22-46b2-a761-839a636062bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6501053893ad408db9e647e5ce3b2afe)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:11.111634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bdbc2-6d22-46b2-a761-839a636062bb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:11.134955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:11.180731 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:11.213150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf6bdbc2-6d22-46b2-a761-839a636062bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6501053893ad408db9e647e5ce3b2afe'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:11.213857 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf6bdbc2-6d22-46b2-a761-839a636062bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6501053893ad408db9e647e5ce3b2afe'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:11.214478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf6bdbc2-6d22-46b2-a761-839a636062bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6501053893ad408db9e647e5ce3b2afe'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:11.233624 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf6bdbc2-6d22-46b2-a761-839a636062bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6501053893ad408db9e647e5ce3b2afe'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:11.247169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215be2a-6d61-4949-9201-fc013f45d0f3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:11.256052 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:11.257345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:11.257345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:11.257645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] PATH_INFO: `/v3/users/c4c9d4ea00444dc7838d2a9c4577192a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:11.258965 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c9d4ea00444dc7838d2a9c4577192a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:11.260136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-bf6bdbc2-6d22-46b2-a761-839a636062bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6501053893ad408db9e647e5ce3b2afe'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:11.260888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:11.285113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 843/1689] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:10 2022] DELETE /identity/v3/users/6501053893ad408db9e647e5ce3b2afe => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:11.290060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eedf64-e5b4-4173-a503-5442f0264c15 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:11.290370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eedf64-e5b4-4173-a503-5442f0264c15 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:11.290627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eedf64-e5b4-4173-a503-5442f0264c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:11.342743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b6eedf64-e5b4-4173-a503-5442f0264c15 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:11.343886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b6eedf64-e5b4-4173-a503-5442f0264c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:11.349575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c9d4ea00444dc7838d2a9c4577192a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:11.350287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c9d4ea00444dc7838d2a9c4577192a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:11.381825 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c9d4ea00444dc7838d2a9c4577192a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:11.382673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c9d4ea00444dc7838d2a9c4577192a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:11.401069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c9d4ea00444dc7838d2a9c4577192a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:11.427125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 847/1690] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:10 2022] DELETE /identity/v3/users/c4c9d4ea00444dc7838d2a9c4577192a => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:11.463319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 844/1691] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:11.500659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:11.502999 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:11.606501 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2065e9-9483-4163-a28c-5cfe0c451086 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:11.611430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fbab8d-95e1-4ea3-80c1-68d68860be40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:11.612236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fbab8d-95e1-4ea3-80c1-68d68860be40 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:11.612527 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fbab8d-95e1-4ea3-80c1-68d68860be40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:11.612805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fbab8d-95e1-4ea3-80c1-68d68860be40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:11.748695 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fbab8d-95e1-4ea3-80c1-68d68860be40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:11.751437 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fbab8d-95e1-4ea3-80c1-68d68860be40 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:11.781064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 848/1692] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:11.921118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eedf64-e5b4-4173-a503-5442f0264c15 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:11.923776 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b6eedf64-e5b4-4173-a503-5442f0264c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:11.948593 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eedf64-e5b4-4173-a503-5442f0264c15 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:11.966846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:11.969022 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:11.969312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:11.969587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] PATH_INFO: `/v3/projects/5833854955b349f282ddca21961a9068` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:11.978437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5833854955b349f282ddca21961a9068)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:11.982868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:12.095945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5833854955b349f282ddca21961a9068', 'request_id': 'req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:12.112831 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5833854955b349f282ddca21961a9068', 'request_id': 'req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:12.133415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5833854955b349f282ddca21961a9068', 'request_id': 'req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:12.174973 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] The token cache is being invalidate because project 5833854955b349f282ddca21961a9068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:12.175268 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:12.177872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:12.185823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 845/1693] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:11 2022] DELETE /identity/v3/projects/5833854955b349f282ddca21961a9068 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:12.187268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e9802-3797-451f-98f8-6af2bf4803cd None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:12.187716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e9802-3797-451f-98f8-6af2bf4803cd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:12.187806 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e9802-3797-451f-98f8-6af2bf4803cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:12.207055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:12.260487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d26e9802-3797-451f-98f8-6af2bf4803cd None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:12.261913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d26e9802-3797-451f-98f8-6af2bf4803cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:12.281815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:12.285685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:12.317927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:12.446419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 849/1694] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:12.455271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0743b-e039-4908-985a-3031e46b7b57 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:12.455651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0743b-e039-4908-985a-3031e46b7b57 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:12.455826 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0743b-e039-4908-985a-3031e46b7b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:12.522056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-bfc0743b-e039-4908-985a-3031e46b7b57 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:12.523313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bfc0743b-e039-4908-985a-3031e46b7b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:12.536911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee83d6-a9ef-4f0a-b140-80e8d5106aaf admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:12.550282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 850/1695] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:43:12.553746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0743b-e039-4908-985a-3031e46b7b57 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:12.557959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:12.562932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:12.563229 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:12.563597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] PATH_INFO: `/v3/projects/f9bb360edfae4d37830cd340ae25b59a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:12.566984 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9bb360edfae4d37830cd340ae25b59a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:12.570121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:12.572381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0743b-e039-4908-985a-3031e46b7b57 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:12.587497 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7f657-9cda-4303-b977-6c9ec6f86792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:12.588531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7f657-9cda-4303-b977-6c9ec6f86792 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:12.589192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7f657-9cda-4303-b977-6c9ec6f86792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:12.589833 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7f657-9cda-4303-b977-6c9ec6f86792 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:12.590903 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7f657-9cda-4303-b977-6c9ec6f86792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:12.591431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7f657-9cda-4303-b977-6c9ec6f86792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:12.593859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7f657-9cda-4303-b977-6c9ec6f86792 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:12.662336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 851/1696] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:43:12.663652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7f657-9cda-4303-b977-6c9ec6f86792 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:12.699882 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7f657-9cda-4303-b977-6c9ec6f86792 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:12.712159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cac53-11f1-4e22-916d-f5c961b1c76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:12.715458 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cac53-11f1-4e22-916d-f5c961b1c76d admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:12.716021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cac53-11f1-4e22-916d-f5c961b1c76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:12.716579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cac53-11f1-4e22-916d-f5c961b1c76d admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:12.717290 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79cac53-11f1-4e22-916d-f5c961b1c76d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:12.723542 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79cac53-11f1-4e22-916d-f5c961b1c76d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:12.742583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9bb360edfae4d37830cd340ae25b59a', 'request_id': 'req-b44746f4-4126-4312-a0ce-50c1670db891'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:12.749706 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9bb360edfae4d37830cd340ae25b59a', 'request_id': 'req-b44746f4-4126-4312-a0ce-50c1670db891'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:12.759037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9bb360edfae4d37830cd340ae25b59a', 'request_id': 'req-b44746f4-4126-4312-a0ce-50c1670db891'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:12.776771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] The token cache is being invalidate because project f9bb360edfae4d37830cd340ae25b59a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:12.777515 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:12.778826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b44746f4-4126-4312-a0ce-50c1670db891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:12.785303 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 846/1697] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:12 2022] DELETE /identity/v3/projects/f9bb360edfae4d37830cd340ae25b59a => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:12.789528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 852/1698] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:12 2022] POST /identity/v3/projects => generated 391 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:12.789693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee85d8e-84b2-4aea-a7c7-5242be597861 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:12.789693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee85d8e-84b2-4aea-a7c7-5242be597861 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:12.789693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee85d8e-84b2-4aea-a7c7-5242be597861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:12.790595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cac53-11f1-4e22-916d-f5c961b1c76d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:12.794354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d79cac53-11f1-4e22-916d-f5c961b1c76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:12.811036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2ee85d8e-84b2-4aea-a7c7-5242be597861 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:12.812365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2ee85d8e-84b2-4aea-a7c7-5242be597861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:12.951026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79cac53-11f1-4e22-916d-f5c961b1c76d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:12.960383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeb5dc-5855-4cd0-ae42-1810b5feb400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:12.961021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeb5dc-5855-4cd0-ae42-1810b5feb400 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:12.961336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeb5dc-5855-4cd0-ae42-1810b5feb400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:12.961611 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeb5dc-5855-4cd0-ae42-1810b5feb400 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:12.962291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaeb5dc-5855-4cd0-ae42-1810b5feb400 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:12.964557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaeb5dc-5855-4cd0-ae42-1810b5feb400 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:12.977081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 847/1699] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:12.979306 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf558d7-db52-4b47-b811-930d1191c2da None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:12.979597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf558d7-db52-4b47-b811-930d1191c2da None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:12.979980 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf558d7-db52-4b47-b811-930d1191c2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.016070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 853/1700] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:12 2022] POST /identity/v3/users => generated 296 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:43:13.019040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeb5dc-5855-4cd0-ae42-1810b5feb400 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.043151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4bf558d7-db52-4b47-b811-930d1191c2da None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:13.048245 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4bf558d7-db52-4b47-b811-930d1191c2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:13.063504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeb5dc-5855-4cd0-ae42-1810b5feb400 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.078284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 848/1701] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:13.081562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf558d7-db52-4b47-b811-930d1191c2da None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.084443 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0eaae-abf3-4689-aa7e-fde5dd7291cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.086391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0eaae-abf3-4689-aa7e-fde5dd7291cb admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.088488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0eaae-abf3-4689-aa7e-fde5dd7291cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.089052 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0eaae-abf3-4689-aa7e-fde5dd7291cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.089969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0eaae-abf3-4689-aa7e-fde5dd7291cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.094184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0eaae-abf3-4689-aa7e-fde5dd7291cb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.104260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf558d7-db52-4b47-b811-930d1191c2da None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.112864 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d1b29-8499-4230-9d59-c9450f9ac0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.116449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d1b29-8499-4230-9d59-c9450f9ac0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.116812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d1b29-8499-4230-9d59-c9450f9ac0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.118134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d1b29-8499-4230-9d59-c9450f9ac0a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.119014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d1b29-8499-4230-9d59-c9450f9ac0a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:13.120001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d1b29-8499-4230-9d59-c9450f9ac0a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.125116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 854/1702] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:13.126593 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d1b29-8499-4230-9d59-c9450f9ac0a5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.130512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0eaae-abf3-4689-aa7e-fde5dd7291cb admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.147802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0eaae-abf3-4689-aa7e-fde5dd7291cb admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.149269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 849/1703] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:13.161728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86472cf2-42dc-44af-850b-5d9674565870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.162697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86472cf2-42dc-44af-850b-5d9674565870 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.163180 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86472cf2-42dc-44af-850b-5d9674565870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.163626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86472cf2-42dc-44af-850b-5d9674565870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.164060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d1b29-8499-4230-9d59-c9450f9ac0a5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.165196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86472cf2-42dc-44af-850b-5d9674565870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.168841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86472cf2-42dc-44af-850b-5d9674565870 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.190396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 855/1704] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:43:13.191014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d1b29-8499-4230-9d59-c9450f9ac0a5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.194507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86472cf2-42dc-44af-850b-5d9674565870 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.195921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e702d-15f5-460b-95b1-434620b13c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.196734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e702d-15f5-460b-95b1-434620b13c02 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.197017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e702d-15f5-460b-95b1-434620b13c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.197293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e702d-15f5-460b-95b1-434620b13c02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.198021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e702d-15f5-460b-95b1-434620b13c02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.199981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e702d-15f5-460b-95b1-434620b13c02 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.211902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86472cf2-42dc-44af-850b-5d9674565870 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.219193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3c1f0-01e6-472b-b6e4-e2f58a389d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.219883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3c1f0-01e6-472b-b6e4-e2f58a389d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.221375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3c1f0-01e6-472b-b6e4-e2f58a389d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.221649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3c1f0-01e6-472b-b6e4-e2f58a389d19 admin admin] PATH_INFO: `/v3/projects/37501c69006d462590b6c5e81e8911c4/users/c2c67b34ecd34d1e8fbc56d7a6e237c2/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.241629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:13.255151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 850/1705] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:13 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:13.256106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e702d-15f5-460b-95b1-434620b13c02 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.286725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3c1f0-01e6-472b-b6e4-e2f58a389d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37501c69006d462590b6c5e81e8911c4, user_id=c2c67b34ecd34d1e8fbc56d7a6e237c2, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.289916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3c1f0-01e6-472b-b6e4-e2f58a389d19 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.329189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 856/1706] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:13 2022] PUT /identity/v3/projects/37501c69006d462590b6c5e81e8911c4/users/c2c67b34ecd34d1e8fbc56d7a6e237c2/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:13.340068 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3bf1e-8b00-4bb2-8888-2766fcb63d5f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.340898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3bf1e-8b00-4bb2-8888-2766fcb63d5f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.341358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3bf1e-8b00-4bb2-8888-2766fcb63d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.351682 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:13.383395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3de3bf1e-8b00-4bb2-8888-2766fcb63d5f None None] MFA Rules not processed for user `c2c67b34ecd34d1e8fbc56d7a6e237c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:13.384877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3de3bf1e-8b00-4bb2-8888-2766fcb63d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:13.421563 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e702d-15f5-460b-95b1-434620b13c02 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.524423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e277f2-cbd6-40b1-9a75-1bb82f68bfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.525064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e277f2-cbd6-40b1-9a75-1bb82f68bfd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.525844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e277f2-cbd6-40b1-9a75-1bb82f68bfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.525844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e277f2-cbd6-40b1-9a75-1bb82f68bfd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.527316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e277f2-cbd6-40b1-9a75-1bb82f68bfd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.529307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e277f2-cbd6-40b1-9a75-1bb82f68bfd5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.536411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 857/1707] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:13.540898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3bf1e-8b00-4bb2-8888-2766fcb63d5f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.560566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3bf1e-8b00-4bb2-8888-2766fcb63d5f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.566460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32342c3a-7e6a-40b1-87b9-5256ce890b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.567239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32342c3a-7e6a-40b1-87b9-5256ce890b73 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.567588 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32342c3a-7e6a-40b1-87b9-5256ce890b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.567920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32342c3a-7e6a-40b1-87b9-5256ce890b73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.568816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32342c3a-7e6a-40b1-87b9-5256ce890b73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.571717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32342c3a-7e6a-40b1-87b9-5256ce890b73 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.583524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 851/1708] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:13 2022] POST /identity/v3/users => generated 288 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:13.588080 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e277f2-cbd6-40b1-9a75-1bb82f68bfd5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.617963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e277f2-cbd6-40b1-9a75-1bb82f68bfd5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.633451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5415e4d-2c2f-4031-bd14-5050853b23c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.636439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5415e4d-2c2f-4031-bd14-5050853b23c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.637098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5415e4d-2c2f-4031-bd14-5050853b23c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.637832 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5415e4d-2c2f-4031-bd14-5050853b23c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.639828 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5415e4d-2c2f-4031-bd14-5050853b23c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.643979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5415e4d-2c2f-4031-bd14-5050853b23c1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.647028 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 858/1709] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:13 2022] POST /identity/v3/projects => generated 393 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:43:13.652142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32342c3a-7e6a-40b1-87b9-5256ce890b73 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.680718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 852/1710] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:13.682457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5415e4d-2c2f-4031-bd14-5050853b23c1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.709073 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5415e4d-2c2f-4031-bd14-5050853b23c1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.771200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32342c3a-7e6a-40b1-87b9-5256ce890b73 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.781393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d741282-77d2-4447-9728-05cac9f41805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.783574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d741282-77d2-4447-9728-05cac9f41805 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.784092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d741282-77d2-4447-9728-05cac9f41805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.784799 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d741282-77d2-4447-9728-05cac9f41805 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.786004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d741282-77d2-4447-9728-05cac9f41805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.789685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d741282-77d2-4447-9728-05cac9f41805 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.843161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146b9f8-5fd8-4788-8945-9bc4f18b0a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.843794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146b9f8-5fd8-4788-8945-9bc4f18b0a6e admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.844067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146b9f8-5fd8-4788-8945-9bc4f18b0a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.844331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146b9f8-5fd8-4788-8945-9bc4f18b0a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.845008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146b9f8-5fd8-4788-8945-9bc4f18b0a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.847051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146b9f8-5fd8-4788-8945-9bc4f18b0a6e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.851431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 859/1711] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:13 2022] POST /identity/v3/users => generated 303 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:13.855796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d741282-77d2-4447-9728-05cac9f41805 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.869030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 853/1712] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:13.875178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d741282-77d2-4447-9728-05cac9f41805 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.875583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146b9f8-5fd8-4788-8945-9bc4f18b0a6e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.884051 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0af9c-0c00-4b5e-857b-6d9be7e3df13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.885681 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0af9c-0c00-4b5e-857b-6d9be7e3df13 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.886874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0af9c-0c00-4b5e-857b-6d9be7e3df13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.887793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0af9c-0c00-4b5e-857b-6d9be7e3df13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.888912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0af9c-0c00-4b5e-857b-6d9be7e3df13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.891110 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0af9c-0c00-4b5e-857b-6d9be7e3df13 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.904859 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146b9f8-5fd8-4788-8945-9bc4f18b0a6e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.909073 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 860/1713] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:43:13.912569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff24aff-7e94-4753-bd3e-bec51ad9251e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.913260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff24aff-7e94-4753-bd3e-bec51ad9251e admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.913919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff24aff-7e94-4753-bd3e-bec51ad9251e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.914218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff24aff-7e94-4753-bd3e-bec51ad9251e admin admin] PATH_INFO: `/v3/projects/887d763e8972497a846556bb6169f42f/users/bc4cdbe2970343cea7119d092de5d189/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.916378 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0af9c-0c00-4b5e-857b-6d9be7e3df13 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:13.941237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff24aff-7e94-4753-bd3e-bec51ad9251e admin admin] RBAC: Authorizing `identity:create_grant(project_id=887d763e8972497a846556bb6169f42f, user_id=bc4cdbe2970343cea7119d092de5d189, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:13.946530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff24aff-7e94-4753-bd3e-bec51ad9251e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:13.946884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0af9c-0c00-4b5e-857b-6d9be7e3df13 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:13.952683 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ddc93-4e5a-4e8a-bf5e-565a933780d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:13.953270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ddc93-4e5a-4e8a-bf5e-565a933780d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.953270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ddc93-4e5a-4e8a-bf5e-565a933780d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.954199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ddc93-4e5a-4e8a-bf5e-565a933780d1 admin admin] PATH_INFO: `/v3/projects/dd6a8c79f2864a7aa7c5e122538c3f54/users/278f8ffce414499595a893dd15313115/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:13.963528 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:13.982121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 854/1714] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:13 2022] PUT /identity/v3/projects/887d763e8972497a846556bb6169f42f/users/bc4cdbe2970343cea7119d092de5d189/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:13.996027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efeafa-ae4c-43a7-ad7c-b8076775796d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:13.996484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efeafa-ae4c-43a7-ad7c-b8076775796d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:13.996610 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efeafa-ae4c-43a7-ad7c-b8076775796d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:14.016554 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ddc93-4e5a-4e8a-bf5e-565a933780d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6a8c79f2864a7aa7c5e122538c3f54, user_id=278f8ffce414499595a893dd15313115, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:14.024343 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ddc93-4e5a-4e8a-bf5e-565a933780d1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:14.053230 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 861/1715] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:13 2022] PUT /identity/v3/projects/dd6a8c79f2864a7aa7c5e122538c3f54/users/278f8ffce414499595a893dd15313115/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:14.058158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ddc93-4e5a-4e8a-bf5e-565a933780d1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:14.065061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c8efeafa-ae4c-43a7-ad7c-b8076775796d None None] MFA Rules not processed for user `bc4cdbe2970343cea7119d092de5d189`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:14.066478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c8efeafa-ae4c-43a7-ad7c-b8076775796d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:14.128330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:14.209580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 855/1716] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:14.221724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efeafa-ae4c-43a7-ad7c-b8076775796d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:14.223506 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c8efeafa-ae4c-43a7-ad7c-b8076775796d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:14.227365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ddc93-4e5a-4e8a-bf5e-565a933780d1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:14.244097 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b3ccf-0f47-4d93-834a-d4fa5a3e1ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:14.245596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b3ccf-0f47-4d93-834a-d4fa5a3e1ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:14.245596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b3ccf-0f47-4d93-834a-d4fa5a3e1ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:14.256639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b3ccf-0f47-4d93-834a-d4fa5a3e1ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:14.257247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b3ccf-0f47-4d93-834a-d4fa5a3e1ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:14.260045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b3ccf-0f47-4d93-834a-d4fa5a3e1ee3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:14.281017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 862/1717] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:14.281017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b3ccf-0f47-4d93-834a-d4fa5a3e1ee3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:14.281665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2d1b3ccf-0f47-4d93-834a-d4fa5a3e1ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:14.303809 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b3ccf-0f47-4d93-834a-d4fa5a3e1ee3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:14.309680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0317709-8d6e-4f69-bc21-4d4dcf7575c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:14.310290 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0317709-8d6e-4f69-bc21-4d4dcf7575c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:14.310687 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0317709-8d6e-4f69-bc21-4d4dcf7575c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:14.310964 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0317709-8d6e-4f69-bc21-4d4dcf7575c1 admin admin] PATH_INFO: `/v3/users/d317698f009d40f89953c40969083f8d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:14.312405 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0317709-8d6e-4f69-bc21-4d4dcf7575c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d317698f009d40f89953c40969083f8d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:14.315252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0317709-8d6e-4f69-bc21-4d4dcf7575c1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:14.322546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efeafa-ae4c-43a7-ad7c-b8076775796d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:14.327839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456de68f-58c1-44b3-b12c-df859b3753fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:14.329016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456de68f-58c1-44b3-b12c-df859b3753fe service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:14.329429 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456de68f-58c1-44b3-b12c-df859b3753fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:14.329776 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456de68f-58c1-44b3-b12c-df859b3753fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:14.340261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456de68f-58c1-44b3-b12c-df859b3753fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:14.341929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456de68f-58c1-44b3-b12c-df859b3753fe service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:14.357487 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 856/1718] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:14.359161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456de68f-58c1-44b3-b12c-df859b3753fe service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:14.398057 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456de68f-58c1-44b3-b12c-df859b3753fe service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:14.403643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44769d63-5c2d-4097-a802-b34174fd97b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:14.404466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44769d63-5c2d-4097-a802-b34174fd97b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:14.404748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44769d63-5c2d-4097-a802-b34174fd97b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:14.405043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44769d63-5c2d-4097-a802-b34174fd97b3 admin admin] PATH_INFO: `/v3/projects/dd6a8c79f2864a7aa7c5e122538c3f54/users/278f8ffce414499595a893dd15313115/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:14.407007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44769d63-5c2d-4097-a802-b34174fd97b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6a8c79f2864a7aa7c5e122538c3f54, user_id=278f8ffce414499595a893dd15313115, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:14.407309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-e0317709-8d6e-4f69-bc21-4d4dcf7575c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd317698f009d40f89953c40969083f8d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:14.410265 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-e0317709-8d6e-4f69-bc21-4d4dcf7575c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd317698f009d40f89953c40969083f8d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:14.410919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44769d63-5c2d-4097-a802-b34174fd97b3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:14.440968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-e0317709-8d6e-4f69-bc21-4d4dcf7575c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd317698f009d40f89953c40969083f8d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:14.441893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-e0317709-8d6e-4f69-bc21-4d4dcf7575c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd317698f009d40f89953c40969083f8d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:14.456902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 857/1719] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:14 2022] PUT /identity/v3/projects/dd6a8c79f2864a7aa7c5e122538c3f54/users/278f8ffce414499595a893dd15313115/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:14.458695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43dea7-d030-4972-826d-a140ffbecdcd None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:14.459787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43dea7-d030-4972-826d-a140ffbecdcd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:14.459952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43dea7-d030-4972-826d-a140ffbecdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:14.464591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-e0317709-8d6e-4f69-bc21-4d4dcf7575c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd317698f009d40f89953c40969083f8d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:14.494929 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 863/1720] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:14 2022] DELETE /identity/v3/users/d317698f009d40f89953c40969083f8d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:14.501338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6fc8e-93ea-49c2-a17a-b1f136d567b4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:14.502478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6fc8e-93ea-49c2-a17a-b1f136d567b4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:14.502763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6fc8e-93ea-49c2-a17a-b1f136d567b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:14.506822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2e43dea7-d030-4972-826d-a140ffbecdcd None None] MFA Rules not processed for user `278f8ffce414499595a893dd15313115`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:14.512198 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2e43dea7-d030-4972-826d-a140ffbecdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:14.540621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9fa6fc8e-93ea-49c2-a17a-b1f136d567b4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:14.541873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9fa6fc8e-93ea-49c2-a17a-b1f136d567b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:14.661328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 864/1721] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:14.665008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 858/1722] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:14 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:14.668960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6fc8e-93ea-49c2-a17a-b1f136d567b4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:14.671319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9fa6fc8e-93ea-49c2-a17a-b1f136d567b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:14.697121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43dea7-d030-4972-826d-a140ffbecdcd None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:14.699348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2e43dea7-d030-4972-826d-a140ffbecdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:14.705850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:14.728926 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:14.751529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6fc8e-93ea-49c2-a17a-b1f136d567b4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:14.756009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39c0d3-3def-4e66-a03f-ccc2824afaee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:14.756593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39c0d3-3def-4e66-a03f-ccc2824afaee service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:14.756868 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39c0d3-3def-4e66-a03f-ccc2824afaee service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:14.757136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39c0d3-3def-4e66-a03f-ccc2824afaee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:14.768256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39c0d3-3def-4e66-a03f-ccc2824afaee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:14.769726 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39c0d3-3def-4e66-a03f-ccc2824afaee service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:14.776079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43dea7-d030-4972-826d-a140ffbecdcd None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:14.779548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 865/1723] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:14.787759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6689f81-ad8f-4011-b4e0-7916625bfd18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:14.789411 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6689f81-ad8f-4011-b4e0-7916625bfd18 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:14.789959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6689f81-ad8f-4011-b4e0-7916625bfd18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:14.790520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6689f81-ad8f-4011-b4e0-7916625bfd18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:14.805710 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6689f81-ad8f-4011-b4e0-7916625bfd18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:14.807800 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6689f81-ad8f-4011-b4e0-7916625bfd18 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:14.822932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 859/1724] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:14 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:43:14.892773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39c0d3-3def-4e66-a03f-ccc2824afaee service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:14.894703 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2d39c0d3-3def-4e66-a03f-ccc2824afaee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:14.909272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39c0d3-3def-4e66-a03f-ccc2824afaee service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:14.913920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996e028-dfb4-4472-adfd-3342332c0837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:14.914109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996e028-dfb4-4472-adfd-3342332c0837 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:14.914427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996e028-dfb4-4472-adfd-3342332c0837 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:14.914757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996e028-dfb4-4472-adfd-3342332c0837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:14.916899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e996e028-dfb4-4472-adfd-3342332c0837 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:14.986520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996e028-dfb4-4472-adfd-3342332c0837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:14.987976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996e028-dfb4-4472-adfd-3342332c0837 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:14.997246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 866/1725] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:15.109079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6689f81-ad8f-4011-b4e0-7916625bfd18 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:15.110620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e6689f81-ad8f-4011-b4e0-7916625bfd18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:15.141223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6689f81-ad8f-4011-b4e0-7916625bfd18 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:15.148806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:15.149721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:15.150467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:15.150969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] PATH_INFO: `/v3/projects/5a91009b85ad4d1195ed75d0bfe853de` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:15.152225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a91009b85ad4d1195ed75d0bfe853de)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:15.154137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:15.223542 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a91009b85ad4d1195ed75d0bfe853de', 'request_id': 'req-c8b3ac9f-4359-4c73-9521-10bd75044dc6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.231300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a91009b85ad4d1195ed75d0bfe853de', 'request_id': 'req-c8b3ac9f-4359-4c73-9521-10bd75044dc6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.239089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a91009b85ad4d1195ed75d0bfe853de', 'request_id': 'req-c8b3ac9f-4359-4c73-9521-10bd75044dc6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.261929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] The token cache is being invalidate because project 5a91009b85ad4d1195ed75d0bfe853de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:15.262257 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.263191 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.267570 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 860/1726] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:15 2022] DELETE /identity/v3/projects/5a91009b85ad4d1195ed75d0bfe853de => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:15.271997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996e028-dfb4-4472-adfd-3342332c0837 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:15.290481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996e028-dfb4-4472-adfd-3342332c0837 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:15.443096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a7f15-d96c-41a8-b8f8-644cbffb1405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:15.444132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7f15-d96c-41a8-b8f8-644cbffb1405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:15.444885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7f15-d96c-41a8-b8f8-644cbffb1405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:15.445383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7f15-d96c-41a8-b8f8-644cbffb1405 admin admin] PATH_INFO: `/v3/users/32365633b27845d09d7f13f221e7a1e8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:15.447434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a7f15-d96c-41a8-b8f8-644cbffb1405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32365633b27845d09d7f13f221e7a1e8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:15.449715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a7f15-d96c-41a8-b8f8-644cbffb1405 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:15.501653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a62a7f15-d96c-41a8-b8f8-644cbffb1405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32365633b27845d09d7f13f221e7a1e8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.502280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a62a7f15-d96c-41a8-b8f8-644cbffb1405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32365633b27845d09d7f13f221e7a1e8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.514014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a62a7f15-d96c-41a8-b8f8-644cbffb1405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32365633b27845d09d7f13f221e7a1e8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.514712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a62a7f15-d96c-41a8-b8f8-644cbffb1405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32365633b27845d09d7f13f221e7a1e8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.523237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a62a7f15-d96c-41a8-b8f8-644cbffb1405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32365633b27845d09d7f13f221e7a1e8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.526781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:15.538278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 867/1727] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:15 2022] DELETE /identity/v3/users/32365633b27845d09d7f13f221e7a1e8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:15.831118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:15.832672 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:15.846486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3ac9f-4359-4c73-9521-10bd75044dc6 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:15.915558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:15.916155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:15.916467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:15.916758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] PATH_INFO: `/v3/projects/d36ab2a3e2d14349b2bea21370c91da3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:15.918073 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d36ab2a3e2d14349b2bea21370c91da3)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:15.920139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:15.959839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:15.976162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd36ab2a3e2d14349b2bea21370c91da3', 'request_id': 'req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.984444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd36ab2a3e2d14349b2bea21370c91da3', 'request_id': 'req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:15.991450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd36ab2a3e2d14349b2bea21370c91da3', 'request_id': 'req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:16.005069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] The token cache is being invalidate because project d36ab2a3e2d14349b2bea21370c91da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:16.005362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:16.006021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b6377fca-6dc9-4055-8da8-5f1ecb9a8d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:16.007549 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 861/1728] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:15 2022] DELETE /identity/v3/projects/d36ab2a3e2d14349b2bea21370c91da3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:16.013381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcefd6-4b26-4817-8e3c-e73955472bd3 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:16.013671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcefd6-4b26-4817-8e3c-e73955472bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:16.013949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcefd6-4b26-4817-8e3c-e73955472bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:16.035303 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5bbcefd6-4b26-4817-8e3c-e73955472bd3 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:16.036684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5bbcefd6-4b26-4817-8e3c-e73955472bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:16.102015 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 868/1729] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:16.110057 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4064af-3956-496d-87ab-c84c9aab28ef None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:16.110548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4064af-3956-496d-87ab-c84c9aab28ef None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:16.110699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4064af-3956-496d-87ab-c84c9aab28ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:16.133837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-bd4064af-3956-496d-87ab-c84c9aab28ef None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:16.135100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bd4064af-3956-496d-87ab-c84c9aab28ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:16.145234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 862/1730] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:16.149485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcefd6-4b26-4817-8e3c-e73955472bd3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:16.159403 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcefd6-4b26-4817-8e3c-e73955472bd3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:16.164061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61564-8532-4b77-ad17-1ec630c4b208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:16.164713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61564-8532-4b77-ad17-1ec630c4b208 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:16.165031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61564-8532-4b77-ad17-1ec630c4b208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:16.165362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61564-8532-4b77-ad17-1ec630c4b208 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:16.166042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a61564-8532-4b77-ad17-1ec630c4b208 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:16.166377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a61564-8532-4b77-ad17-1ec630c4b208 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:16.167945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a61564-8532-4b77-ad17-1ec630c4b208 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:16.184438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 869/1731] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:16.185637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4064af-3956-496d-87ab-c84c9aab28ef None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:16.197520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4064af-3956-496d-87ab-c84c9aab28ef None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:16.202572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40e03c-ae48-443e-bee7-45b0fb115109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:16.203331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40e03c-ae48-443e-bee7-45b0fb115109 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:16.203678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40e03c-ae48-443e-bee7-45b0fb115109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:16.204015 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40e03c-ae48-443e-bee7-45b0fb115109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:16.205294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40e03c-ae48-443e-bee7-45b0fb115109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:16.207200 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40e03c-ae48-443e-bee7-45b0fb115109 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:16.243461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 863/1732] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:16 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:16.248357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61564-8532-4b77-ad17-1ec630c4b208 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:16.260239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61564-8532-4b77-ad17-1ec630c4b208 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:16.328889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f8911-47e9-4004-aa58-ee408f0f67fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:16.330790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8911-47e9-4004-aa58-ee408f0f67fb admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:16.331117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8911-47e9-4004-aa58-ee408f0f67fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:16.331519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f8911-47e9-4004-aa58-ee408f0f67fb admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:16.332167 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f8911-47e9-4004-aa58-ee408f0f67fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:16.334123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f8911-47e9-4004-aa58-ee408f0f67fb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:16.359431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 870/1733] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:16 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:16.364093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40e03c-ae48-443e-bee7-45b0fb115109 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:16.382502 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40e03c-ae48-443e-bee7-45b0fb115109 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:16.387168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a40aa8-92ed-440a-ab38-3fe0098824b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:16.387915 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a40aa8-92ed-440a-ab38-3fe0098824b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:16.388195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a40aa8-92ed-440a-ab38-3fe0098824b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:16.388463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a40aa8-92ed-440a-ab38-3fe0098824b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:16.389101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a40aa8-92ed-440a-ab38-3fe0098824b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:16.390734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a40aa8-92ed-440a-ab38-3fe0098824b4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:16.404584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 864/1734] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:16.408966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f8911-47e9-4004-aa58-ee408f0f67fb admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:16.419433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f8911-47e9-4004-aa58-ee408f0f67fb admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:16.425615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c454e-598b-4732-be86-c7c2e2dd3efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:16.426622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c454e-598b-4732-be86-c7c2e2dd3efe admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:16.427115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c454e-598b-4732-be86-c7c2e2dd3efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:16.427564 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c454e-598b-4732-be86-c7c2e2dd3efe admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:16.428250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c454e-598b-4732-be86-c7c2e2dd3efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:16.429858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c454e-598b-4732-be86-c7c2e2dd3efe admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:16.441769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 871/1735] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:16.445700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a40aa8-92ed-440a-ab38-3fe0098824b4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:16.457158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a40aa8-92ed-440a-ab38-3fe0098824b4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:16.465228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87c3d1-9ba7-46d5-8710-065bee29398d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:16.467494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c3d1-9ba7-46d5-8710-065bee29398d admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:16.467820 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c3d1-9ba7-46d5-8710-065bee29398d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:16.468196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87c3d1-9ba7-46d5-8710-065bee29398d admin admin] PATH_INFO: `/v3/projects/c21205d82ebe43fcad528652e95173a4/users/158496fb8bbd4460bb966a02645105ff/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:16.486863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87c3d1-9ba7-46d5-8710-065bee29398d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21205d82ebe43fcad528652e95173a4, user_id=158496fb8bbd4460bb966a02645105ff, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:16.490947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87c3d1-9ba7-46d5-8710-065bee29398d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:16.509838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 865/1736] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:16 2022] PUT /identity/v3/projects/c21205d82ebe43fcad528652e95173a4/users/158496fb8bbd4460bb966a02645105ff/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:16.522490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d60a2-99fb-41af-ac23-d7aad1d364fc None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:16.522894 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d60a2-99fb-41af-ac23-d7aad1d364fc None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:16.523180 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d60a2-99fb-41af-ac23-d7aad1d364fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:16.594843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d10d60a2-99fb-41af-ac23-d7aad1d364fc None None] MFA Rules not processed for user `158496fb8bbd4460bb966a02645105ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:16.597166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d10d60a2-99fb-41af-ac23-d7aad1d364fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:16.686768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:16.730636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 872/1737] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:16.742214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87c3d1-9ba7-46d5-8710-065bee29398d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:16.743887 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-bc87c3d1-9ba7-46d5-8710-065bee29398d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:16.805043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87c3d1-9ba7-46d5-8710-065bee29398d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:16.808656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b255cca-3998-40c6-ac15-ff5dfb177912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:16.809258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b255cca-3998-40c6-ac15-ff5dfb177912 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:16.809529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b255cca-3998-40c6-ac15-ff5dfb177912 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:16.809791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b255cca-3998-40c6-ac15-ff5dfb177912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:16.819637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b255cca-3998-40c6-ac15-ff5dfb177912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:16.821103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b255cca-3998-40c6-ac15-ff5dfb177912 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:16.831435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 866/1738] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:16.853514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d60a2-99fb-41af-ac23-d7aad1d364fc None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:16.930450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d60a2-99fb-41af-ac23-d7aad1d364fc None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:16.936091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:16.936726 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:16.937003 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:16.937270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] PATH_INFO: `/v3/users/158496fb8bbd4460bb966a02645105ff` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:16.938422 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=158496fb8bbd4460bb966a02645105ff)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:16.940380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:16.996916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '158496fb8bbd4460bb966a02645105ff'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:16.997630 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '158496fb8bbd4460bb966a02645105ff'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.013557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '158496fb8bbd4460bb966a02645105ff'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.014377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '158496fb8bbd4460bb966a02645105ff'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.025924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '158496fb8bbd4460bb966a02645105ff'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.041581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 873/1739] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:16 2022] DELETE /identity/v3/users/158496fb8bbd4460bb966a02645105ff => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:17.046151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6bb64-9450-428c-9bed-90c179f71f73 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:17.046470 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6bb64-9450-428c-9bed-90c179f71f73 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:17.046794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6bb64-9450-428c-9bed-90c179f71f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:17.068581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-01b6bb64-9450-428c-9bed-90c179f71f73 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:17.070003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-01b6bb64-9450-428c-9bed-90c179f71f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:17.141557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:17.154446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 867/1740] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:43:17.162027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:17.167304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:17.254704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e5426-00b6-4891-91d2-4fbb2dd150bc admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:17.258544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909b523-ddfc-476c-bcbf-49a2a4a5f339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:17.259267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909b523-ddfc-476c-bcbf-49a2a4a5f339 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:17.259656 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909b523-ddfc-476c-bcbf-49a2a4a5f339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:17.259914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909b523-ddfc-476c-bcbf-49a2a4a5f339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:17.271689 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909b523-ddfc-476c-bcbf-49a2a4a5f339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:17.273136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909b523-ddfc-476c-bcbf-49a2a4a5f339 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:17.284736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 874/1741] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:17.567132 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6bb64-9450-428c-9bed-90c179f71f73 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:17.568597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-01b6bb64-9450-428c-9bed-90c179f71f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:17.581314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6bb64-9450-428c-9bed-90c179f71f73 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:17.586210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:17.586804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:17.587085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:17.587359 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] PATH_INFO: `/v3/projects/c21205d82ebe43fcad528652e95173a4` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:17.588514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c21205d82ebe43fcad528652e95173a4)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:17.590599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:17.636649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c21205d82ebe43fcad528652e95173a4', 'request_id': 'req-b2ad14a7-998a-432a-ba29-b991ed55e767'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.644459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c21205d82ebe43fcad528652e95173a4', 'request_id': 'req-b2ad14a7-998a-432a-ba29-b991ed55e767'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.650665 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c21205d82ebe43fcad528652e95173a4', 'request_id': 'req-b2ad14a7-998a-432a-ba29-b991ed55e767'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.661294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] The token cache is being invalidate because project c21205d82ebe43fcad528652e95173a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:17.661579 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.662222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.663729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 868/1742] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:17 2022] DELETE /identity/v3/projects/c21205d82ebe43fcad528652e95173a4 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:17.668793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcc913-c649-4af0-8fb5-12d145d8e6ee None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:17.669312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcc913-c649-4af0-8fb5-12d145d8e6ee None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:17.669312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcc913-c649-4af0-8fb5-12d145d8e6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:17.689190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f0bcc913-c649-4af0-8fb5-12d145d8e6ee None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:17.690499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f0bcc913-c649-4af0-8fb5-12d145d8e6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:17.699666 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:17.761381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:17.763214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:17.773407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 875/1743] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:17.775552 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208d9f-cfdb-41d4-967b-ec4a1308c2ed None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:17.776550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208d9f-cfdb-41d4-967b-ec4a1308c2ed None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:17.776938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208d9f-cfdb-41d4-967b-ec4a1308c2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:17.782994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad14a7-998a-432a-ba29-b991ed55e767 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:17.795580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:17.796551 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:17.796869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:17.797175 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] PATH_INFO: `/v3/users/bc4cdbe2970343cea7119d092de5d189` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:17.798476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc4cdbe2970343cea7119d092de5d189)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:17.799132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a5208d9f-cfdb-41d4-967b-ec4a1308c2ed None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:17.800466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a5208d9f-cfdb-41d4-967b-ec4a1308c2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:17.800588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:17.810829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 876/1744] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:17.815991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5208d9f-cfdb-41d4-967b-ec4a1308c2ed None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:17.828781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5208d9f-cfdb-41d4-967b-ec4a1308c2ed None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:17.835124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811bd53-7fb4-4f57-af25-a745b7db10e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:17.836717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811bd53-7fb4-4f57-af25-a745b7db10e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:17.836934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811bd53-7fb4-4f57-af25-a745b7db10e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:17.837212 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811bd53-7fb4-4f57-af25-a745b7db10e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:17.837920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811bd53-7fb4-4f57-af25-a745b7db10e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:17.838382 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811bd53-7fb4-4f57-af25-a745b7db10e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:17.840131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811bd53-7fb4-4f57-af25-a745b7db10e1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:17.854474 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 877/1745] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:17.858941 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811bd53-7fb4-4f57-af25-a745b7db10e1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:17.864407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc4cdbe2970343cea7119d092de5d189'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.865330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc4cdbe2970343cea7119d092de5d189'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.866150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc4cdbe2970343cea7119d092de5d189'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.871156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811bd53-7fb4-4f57-af25-a745b7db10e1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:17.874902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc4cdbe2970343cea7119d092de5d189'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.876613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e494f8-185a-46c5-8c32-42326df0477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:17.877309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e494f8-185a-46c5-8c32-42326df0477f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:17.877716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e494f8-185a-46c5-8c32-42326df0477f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:17.877996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e494f8-185a-46c5-8c32-42326df0477f admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:17.878673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e494f8-185a-46c5-8c32-42326df0477f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:17.880526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e494f8-185a-46c5-8c32-42326df0477f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:17.891374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc4cdbe2970343cea7119d092de5d189'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:17.912891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 878/1747] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:17 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:43:17.916236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 869/1747] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:17 2022] DELETE /identity/v3/users/bc4cdbe2970343cea7119d092de5d189 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:17.916353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28fe42-886d-46d9-b49c-0558c3cfacd2 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:17.916353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28fe42-886d-46d9-b49c-0558c3cfacd2 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:17.916353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28fe42-886d-46d9-b49c-0558c3cfacd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:17.917792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:17.919835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:17.936816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9a28fe42-886d-46d9-b49c-0558c3cfacd2 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:17.938259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9a28fe42-886d-46d9-b49c-0558c3cfacd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:17.976713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:17.993745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0de21-b2b9-4a5b-9d61-a93998f7c687 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:18.005781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 879/1748] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:18.015442 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28fe42-886d-46d9-b49c-0558c3cfacd2 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:18.016830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9a28fe42-886d-46d9-b49c-0558c3cfacd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:18.058558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb86041-b372-4596-9838-4c3748bad002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:18.059304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb86041-b372-4596-9838-4c3748bad002 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.059587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb86041-b372-4596-9838-4c3748bad002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.059863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb86041-b372-4596-9838-4c3748bad002 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.060548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb86041-b372-4596-9838-4c3748bad002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:18.062657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb86041-b372-4596-9838-4c3748bad002 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:18.075174 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28fe42-886d-46d9-b49c-0558c3cfacd2 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:18.079337 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42d386-41ed-4c87-9652-41b0590e357c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:18.080222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42d386-41ed-4c87-9652-41b0590e357c service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.080500 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42d386-41ed-4c87-9652-41b0590e357c service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.080774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42d386-41ed-4c87-9652-41b0590e357c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.085236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 870/1749] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:17 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:18.089502 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb86041-b372-4596-9838-4c3748bad002 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:18.090645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42d386-41ed-4c87-9652-41b0590e357c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:18.092069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42d386-41ed-4c87-9652-41b0590e357c service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:18.098911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb86041-b372-4596-9838-4c3748bad002 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:18.101792 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 880/1750] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:43:18.103606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559267b-6ea8-4f66-a77a-eda2fafe3ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:18.104205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559267b-6ea8-4f66-a77a-eda2fafe3ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.104486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559267b-6ea8-4f66-a77a-eda2fafe3ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.104860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559267b-6ea8-4f66-a77a-eda2fafe3ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.105501 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559267b-6ea8-4f66-a77a-eda2fafe3ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:18.107116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559267b-6ea8-4f66-a77a-eda2fafe3ab3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:18.118307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 871/1751] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:43:18.122346 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42d386-41ed-4c87-9652-41b0590e357c service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:18.132537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42d386-41ed-4c87-9652-41b0590e357c service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:18.137167 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377ddc2-67b1-4d91-9b0f-3c35f6309604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:18.137764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ddc2-67b1-4d91-9b0f-3c35f6309604 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.138043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ddc2-67b1-4d91-9b0f-3c35f6309604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.138349 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ddc2-67b1-4d91-9b0f-3c35f6309604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.139002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377ddc2-67b1-4d91-9b0f-3c35f6309604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:18.140723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377ddc2-67b1-4d91-9b0f-3c35f6309604 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:18.151565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 881/1752] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:43:18.156253 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559267b-6ea8-4f66-a77a-eda2fafe3ab3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:18.167968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559267b-6ea8-4f66-a77a-eda2fafe3ab3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:18.173489 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf2157-d311-4468-90e1-a78582a1a00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:18.174385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf2157-d311-4468-90e1-a78582a1a00b admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.174659 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf2157-d311-4468-90e1-a78582a1a00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.174932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf2157-d311-4468-90e1-a78582a1a00b admin admin] PATH_INFO: `/v3/projects/be6ef569c3894d2182483d22eab93189/users/6e5a93877d764a8295870cd71033cd24/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.190456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf2157-d311-4468-90e1-a78582a1a00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6ef569c3894d2182483d22eab93189, user_id=6e5a93877d764a8295870cd71033cd24, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:18.193157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf2157-d311-4468-90e1-a78582a1a00b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:18.221339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 872/1753] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:18 2022] PUT /identity/v3/projects/be6ef569c3894d2182483d22eab93189/users/6e5a93877d764a8295870cd71033cd24/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:18.231654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4657-a741-4c75-9e8a-18e4f81b4019 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.232026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4657-a741-4c75-9e8a-18e4f81b4019 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.232302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4657-a741-4c75-9e8a-18e4f81b4019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.242483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:18.271084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-39ef4657-a741-4c75-9e8a-18e4f81b4019 None None] MFA Rules not processed for user `6e5a93877d764a8295870cd71033cd24`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:18.272890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-39ef4657-a741-4c75-9e8a-18e4f81b4019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:18.349827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 882/1754] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:43:18.361472 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf2157-d311-4468-90e1-a78582a1a00b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:18.363358 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fdaf2157-d311-4468-90e1-a78582a1a00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:18.420707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef4657-a741-4c75-9e8a-18e4f81b4019 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:18.422279 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-39ef4657-a741-4c75-9e8a-18e4f81b4019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:18.442993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf2157-d311-4468-90e1-a78582a1a00b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:18.446785 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d377e-2528-4a4f-b956-7f55ef8a0c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:18.447404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d377e-2528-4a4f-b956-7f55ef8a0c04 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.447685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d377e-2528-4a4f-b956-7f55ef8a0c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.448413 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d377e-2528-4a4f-b956-7f55ef8a0c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.460134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d377e-2528-4a4f-b956-7f55ef8a0c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:18.461621 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d377e-2528-4a4f-b956-7f55ef8a0c04 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:18.473402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 873/1755] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:18.526733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef4657-a741-4c75-9e8a-18e4f81b4019 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:18.531486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:18.532141 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.532947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.533479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] PATH_INFO: `/v3/projects/887d763e8972497a846556bb6169f42f` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.534759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=887d763e8972497a846556bb6169f42f)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:18.536707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:18.607040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887d763e8972497a846556bb6169f42f', 'request_id': 'req-c7ad23fe-9770-4b27-a426-ea620eb34b71'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:18.617726 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887d763e8972497a846556bb6169f42f', 'request_id': 'req-c7ad23fe-9770-4b27-a426-ea620eb34b71'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:18.630953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887d763e8972497a846556bb6169f42f', 'request_id': 'req-c7ad23fe-9770-4b27-a426-ea620eb34b71'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:18.648685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] The token cache is being invalidate because project 887d763e8972497a846556bb6169f42f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:18.649103 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:18.649810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c7ad23fe-9770-4b27-a426-ea620eb34b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:18.651381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 883/1756] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:18 2022] DELETE /identity/v3/projects/887d763e8972497a846556bb6169f42f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:18.656374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608375b-65c4-4b46-ab90-4997ee11579a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.656708 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608375b-65c4-4b46-ab90-4997ee11579a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.656991 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608375b-65c4-4b46-ab90-4997ee11579a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.699872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f608375b-65c4-4b46-ab90-4997ee11579a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:18.702153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f608375b-65c4-4b46-ab90-4997ee11579a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:18.760963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:18.779633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa22af-e7c0-4d8c-b303-7d799fd131ce None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.780125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa22af-e7c0-4d8c-b303-7d799fd131ce None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.780217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa22af-e7c0-4d8c-b303-7d799fd131ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.786242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 874/1757] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:18.825776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-70fa22af-e7c0-4d8c-b303-7d799fd131ce None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:18.827348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-70fa22af-e7c0-4d8c-b303-7d799fd131ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:18.839907 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 884/1758] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:18.847475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608375b-65c4-4b46-ab90-4997ee11579a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:18.862452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608375b-65c4-4b46-ab90-4997ee11579a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:18.876282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05780f06-1d67-468a-8fbd-7ee72f78d8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:18.878909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05780f06-1d67-468a-8fbd-7ee72f78d8da admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.879369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05780f06-1d67-468a-8fbd-7ee72f78d8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.879850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05780f06-1d67-468a-8fbd-7ee72f78d8da admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.881334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05780f06-1d67-468a-8fbd-7ee72f78d8da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:18.882102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05780f06-1d67-468a-8fbd-7ee72f78d8da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:18.883957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05780f06-1d67-468a-8fbd-7ee72f78d8da admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:18.891598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa22af-e7c0-4d8c-b303-7d799fd131ce None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:18.893903 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-70fa22af-e7c0-4d8c-b303-7d799fd131ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:18.911123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa22af-e7c0-4d8c-b303-7d799fd131ce None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:18.918844 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36618a22-95f1-453e-b510-2d55d6ff1834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:18.919738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36618a22-95f1-453e-b510-2d55d6ff1834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.920176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36618a22-95f1-453e-b510-2d55d6ff1834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.920659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36618a22-95f1-453e-b510-2d55d6ff1834 admin admin] PATH_INFO: `/v3/users/c2c67b34ecd34d1e8fbc56d7a6e237c2` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.921912 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 875/1759] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:18.922101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36618a22-95f1-453e-b510-2d55d6ff1834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c67b34ecd34d1e8fbc56d7a6e237c2)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:18.924201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36618a22-95f1-453e-b510-2d55d6ff1834 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:18.935588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05780f06-1d67-468a-8fbd-7ee72f78d8da admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:18.940693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:18.946937 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05780f06-1d67-468a-8fbd-7ee72f78d8da admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:18.952064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf56338-7e25-4a5a-9f7f-583afc10e0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:18.952696 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf56338-7e25-4a5a-9f7f-583afc10e0dc admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:18.953003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf56338-7e25-4a5a-9f7f-583afc10e0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:18.953233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf56338-7e25-4a5a-9f7f-583afc10e0dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:18.953828 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf56338-7e25-4a5a-9f7f-583afc10e0dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:18.956476 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf56338-7e25-4a5a-9f7f-583afc10e0dc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:18.991911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-36618a22-95f1-453e-b510-2d55d6ff1834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c67b34ecd34d1e8fbc56d7a6e237c2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:18.992708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-36618a22-95f1-453e-b510-2d55d6ff1834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c67b34ecd34d1e8fbc56d7a6e237c2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:18.994214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 876/1760] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:18 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:43:18.997745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf56338-7e25-4a5a-9f7f-583afc10e0dc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:19.007734 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-36618a22-95f1-453e-b510-2d55d6ff1834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c67b34ecd34d1e8fbc56d7a6e237c2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.009345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-36618a22-95f1-453e-b510-2d55d6ff1834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c67b34ecd34d1e8fbc56d7a6e237c2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.015629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-36618a22-95f1-453e-b510-2d55d6ff1834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c67b34ecd34d1e8fbc56d7a6e237c2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.035950 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 885/1761] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:18 2022] DELETE /identity/v3/users/c2c67b34ecd34d1e8fbc56d7a6e237c2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:19.041081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a7a55-c465-432a-bd66-0be79d37cfbb None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:19.042156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a7a55-c465-432a-bd66-0be79d37cfbb None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:19.042450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a7a55-c465-432a-bd66-0be79d37cfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:19.063276 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-849a7a55-c465-432a-bd66-0be79d37cfbb None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:19.065115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-849a7a55-c465-432a-bd66-0be79d37cfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:19.085582 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf56338-7e25-4a5a-9f7f-583afc10e0dc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:19.142663 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 886/1762] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:19.147630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b511b-8cf1-4f7b-a1b4-bd494e5488f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:19.148284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b511b-8cf1-4f7b-a1b4-bd494e5488f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:19.148562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b511b-8cf1-4f7b-a1b4-bd494e5488f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:19.148859 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b511b-8cf1-4f7b-a1b4-bd494e5488f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:19.149492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b511b-8cf1-4f7b-a1b4-bd494e5488f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:19.151708 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b511b-8cf1-4f7b-a1b4-bd494e5488f4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:19.152324 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a7a55-c465-432a-bd66-0be79d37cfbb None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:19.153687 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-849a7a55-c465-432a-bd66-0be79d37cfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:19.173663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 877/1763] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:18 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:19.177621 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b511b-8cf1-4f7b-a1b4-bd494e5488f4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:19.179390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3a0b511b-8cf1-4f7b-a1b4-bd494e5488f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:19.191601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b511b-8cf1-4f7b-a1b4-bd494e5488f4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:19.198044 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa008e-0418-4ecf-950e-0d7602a19aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:19.198689 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa008e-0418-4ecf-950e-0d7602a19aea admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:19.198966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa008e-0418-4ecf-950e-0d7602a19aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:19.199236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa008e-0418-4ecf-950e-0d7602a19aea admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:19.200241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa008e-0418-4ecf-950e-0d7602a19aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:19.202153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa008e-0418-4ecf-950e-0d7602a19aea admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:19.213281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 878/1764] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:43:19.216948 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa008e-0418-4ecf-950e-0d7602a19aea admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:19.217666 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a7a55-c465-432a-bd66-0be79d37cfbb None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:19.221464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cc5df-5bf5-42af-8dbe-9325c9a3817b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:19.222075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cc5df-5bf5-42af-8dbe-9325c9a3817b service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:19.222387 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cc5df-5bf5-42af-8dbe-9325c9a3817b service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:19.222856 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cc5df-5bf5-42af-8dbe-9325c9a3817b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:19.227594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa008e-0418-4ecf-950e-0d7602a19aea admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:19.232740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec411ae-0480-4d6b-bd8d-346760b9755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:19.233058 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cc5df-5bf5-42af-8dbe-9325c9a3817b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:19.233317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec411ae-0480-4d6b-bd8d-346760b9755d admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:19.233594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec411ae-0480-4d6b-bd8d-346760b9755d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:19.233859 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec411ae-0480-4d6b-bd8d-346760b9755d admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:19.234517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cc5df-5bf5-42af-8dbe-9325c9a3817b service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:19.234612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec411ae-0480-4d6b-bd8d-346760b9755d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:19.236783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec411ae-0480-4d6b-bd8d-346760b9755d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:19.244287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 887/1765] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:43:19.250208 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 879/1766] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:19.252438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cc5df-5bf5-42af-8dbe-9325c9a3817b service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:19.261685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cc5df-5bf5-42af-8dbe-9325c9a3817b service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:19.266254 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed0c86-1649-4306-9eb0-84f20e5bd291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:19.268126 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed0c86-1649-4306-9eb0-84f20e5bd291 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:19.268420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed0c86-1649-4306-9eb0-84f20e5bd291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:19.268686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed0c86-1649-4306-9eb0-84f20e5bd291 admin admin] PATH_INFO: `/v3/projects/b8ee40c2e5304b67b8814d34f3fe97b3/users/687126fda40f481cb5ba5d185cbfa74a/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:19.284321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed0c86-1649-4306-9eb0-84f20e5bd291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ee40c2e5304b67b8814d34f3fe97b3, user_id=687126fda40f481cb5ba5d185cbfa74a, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:19.287056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed0c86-1649-4306-9eb0-84f20e5bd291 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:19.301244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 888/1767] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:19 2022] PUT /identity/v3/projects/b8ee40c2e5304b67b8814d34f3fe97b3/users/687126fda40f481cb5ba5d185cbfa74a/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:19.307834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b55cb2-d94c-4691-b3db-a3fe65f63bb5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:19.308449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b55cb2-d94c-4691-b3db-a3fe65f63bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:19.309027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b55cb2-d94c-4691-b3db-a3fe65f63bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:19.333485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-03b55cb2-d94c-4691-b3db-a3fe65f63bb5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:19.334718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-03b55cb2-d94c-4691-b3db-a3fe65f63bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:19.369909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:19.407399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 880/1768] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:43:19.418097 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed0c86-1649-4306-9eb0-84f20e5bd291 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:19.428666 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:19.476023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed0c86-1649-4306-9eb0-84f20e5bd291 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:19.479716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279346ee-62cf-4dbd-bbd7-7cf841c5cf55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:19.480305 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279346ee-62cf-4dbd-bbd7-7cf841c5cf55 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:19.480587 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279346ee-62cf-4dbd-bbd7-7cf841c5cf55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:19.480861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279346ee-62cf-4dbd-bbd7-7cf841c5cf55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:19.491639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279346ee-62cf-4dbd-bbd7-7cf841c5cf55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:19.493054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279346ee-62cf-4dbd-bbd7-7cf841c5cf55 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:19.503158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 889/1769] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:19.529285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b55cb2-d94c-4691-b3db-a3fe65f63bb5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:19.531134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-03b55cb2-d94c-4691-b3db-a3fe65f63bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:19.554790 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b55cb2-d94c-4691-b3db-a3fe65f63bb5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:19.559560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:19.560199 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:19.560477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:19.560797 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] PATH_INFO: `/v3/projects/37501c69006d462590b6c5e81e8911c4` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:19.562018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=37501c69006d462590b6c5e81e8911c4)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:19.564598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:19.616252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37501c69006d462590b6c5e81e8911c4', 'request_id': 'req-d09e6441-6208-4752-9515-5a342b7ebf7d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.625465 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37501c69006d462590b6c5e81e8911c4', 'request_id': 'req-d09e6441-6208-4752-9515-5a342b7ebf7d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.632275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37501c69006d462590b6c5e81e8911c4', 'request_id': 'req-d09e6441-6208-4752-9515-5a342b7ebf7d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.643866 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] The token cache is being invalidate because project 37501c69006d462590b6c5e81e8911c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:19.644220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.644943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d09e6441-6208-4752-9515-5a342b7ebf7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.646534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 881/1770] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:19 2022] DELETE /identity/v3/projects/37501c69006d462590b6c5e81e8911c4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:19.650544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279346ee-62cf-4dbd-bbd7-7cf841c5cf55 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:19.661532 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279346ee-62cf-4dbd-bbd7-7cf841c5cf55 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:19.749395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:19.750087 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:19.750401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:19.750696 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] PATH_INFO: `/v3/users/278f8ffce414499595a893dd15313115` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:19.752168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] RBAC: Authorizing `identity:delete_user(user_id=278f8ffce414499595a893dd15313115)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:19.754144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:19.805141 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '278f8ffce414499595a893dd15313115'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.805875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '278f8ffce414499595a893dd15313115'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.818570 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '278f8ffce414499595a893dd15313115'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.819273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '278f8ffce414499595a893dd15313115'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.828268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '278f8ffce414499595a893dd15313115'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:19.846546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 890/1771] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:19 2022] DELETE /identity/v3/users/278f8ffce414499595a893dd15313115 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:20.288452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca970006-827e-4577-9b8d-4f9dd61c0b4a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:20.289394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca970006-827e-4577-9b8d-4f9dd61c0b4a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:20.289855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca970006-827e-4577-9b8d-4f9dd61c0b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:20.319252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ca970006-827e-4577-9b8d-4f9dd61c0b4a None None] MFA Rules not processed for user `687126fda40f481cb5ba5d185cbfa74a`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:20.320765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ca970006-827e-4577-9b8d-4f9dd61c0b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:20.383286 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:20.385030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:20.417053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 882/1772] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:20.428618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca970006-827e-4577-9b8d-4f9dd61c0b4a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:20.430406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ca970006-827e-4577-9b8d-4f9dd61c0b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:20.432246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:20.479071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d31b39-da38-4183-b69f-a3923388537d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:20.485694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:20.486636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:20.487041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:20.487316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] PATH_INFO: `/v3/projects/dd6a8c79f2864a7aa7c5e122538c3f54` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:20.488651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd6a8c79f2864a7aa7c5e122538c3f54)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:20.490998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:20.503081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca970006-827e-4577-9b8d-4f9dd61c0b4a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:20.506842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0d5d6-6905-4fa9-9d70-0d0f10114908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:20.507530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0d5d6-6905-4fa9-9d70-0d0f10114908 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:20.507919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0d5d6-6905-4fa9-9d70-0d0f10114908 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:20.508188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0d5d6-6905-4fa9-9d70-0d0f10114908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:20.513370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:20.519105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0d5d6-6905-4fa9-9d70-0d0f10114908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:20.521821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0d5d6-6905-4fa9-9d70-0d0f10114908 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:20.531217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 883/1773] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:43:20.545279 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd6a8c79f2864a7aa7c5e122538c3f54', 'request_id': 'req-d768054b-ff77-44a9-b53e-e1b81d788ea3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:20.550677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd6a8c79f2864a7aa7c5e122538c3f54', 'request_id': 'req-d768054b-ff77-44a9-b53e-e1b81d788ea3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:20.564700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd6a8c79f2864a7aa7c5e122538c3f54', 'request_id': 'req-d768054b-ff77-44a9-b53e-e1b81d788ea3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:20.578246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] The token cache is being invalidate because project dd6a8c79f2864a7aa7c5e122538c3f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:20.578644 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:20.579900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d768054b-ff77-44a9-b53e-e1b81d788ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:20.581925 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 891/1774] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:20 2022] DELETE /identity/v3/projects/dd6a8c79f2864a7aa7c5e122538c3f54 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:20.586970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cd7b4-6ce6-4fcd-bc35-f902312f0d69 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:20.587618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cd7b4-6ce6-4fcd-bc35-f902312f0d69 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:20.588055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cd7b4-6ce6-4fcd-bc35-f902312f0d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:20.609439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d48cd7b4-6ce6-4fcd-bc35-f902312f0d69 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:20.610701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d48cd7b4-6ce6-4fcd-bc35-f902312f0d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:20.678104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 884/1775] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:20.686881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4e5e6-334e-465a-b6be-c559f1034ed4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:20.687190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4e5e6-334e-465a-b6be-c559f1034ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:20.687587 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4e5e6-334e-465a-b6be-c559f1034ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:20.710831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b1c4e5e6-334e-465a-b6be-c559f1034ed4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:20.711969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b1c4e5e6-334e-465a-b6be-c559f1034ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:20.722306 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 892/1776] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:20.725413 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cd7b4-6ce6-4fcd-bc35-f902312f0d69 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:20.735504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cd7b4-6ce6-4fcd-bc35-f902312f0d69 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:20.740811 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5ee76-3768-418b-9c34-8d529bbe2390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:20.741632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5ee76-3768-418b-9c34-8d529bbe2390 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:20.741955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5ee76-3768-418b-9c34-8d529bbe2390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:20.742261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5ee76-3768-418b-9c34-8d529bbe2390 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:20.743051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5ee76-3768-418b-9c34-8d529bbe2390 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:20.743356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5ee76-3768-418b-9c34-8d529bbe2390 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:20.745373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5ee76-3768-418b-9c34-8d529bbe2390 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:20.759498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 885/1777] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:20.764110 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4e5e6-334e-465a-b6be-c559f1034ed4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:20.775010 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4e5e6-334e-465a-b6be-c559f1034ed4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:20.779914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e3df5-8fc3-4a37-bf66-a0a236069477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:20.780520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e3df5-8fc3-4a37-bf66-a0a236069477 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:20.780827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e3df5-8fc3-4a37-bf66-a0a236069477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:20.780827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e3df5-8fc3-4a37-bf66-a0a236069477 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:20.780827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e3df5-8fc3-4a37-bf66-a0a236069477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:20.780827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e3df5-8fc3-4a37-bf66-a0a236069477 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:20.823452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 893/1778] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:20 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:20.831398 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5ee76-3768-418b-9c34-8d529bbe2390 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:20.844902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5ee76-3768-418b-9c34-8d529bbe2390 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:20.906380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc51205-378f-4c5e-94f5-394941dff495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:20.906989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc51205-378f-4c5e-94f5-394941dff495 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:20.907294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc51205-378f-4c5e-94f5-394941dff495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:20.907566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc51205-378f-4c5e-94f5-394941dff495 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:20.908439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc51205-378f-4c5e-94f5-394941dff495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:20.910393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc51205-378f-4c5e-94f5-394941dff495 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:20.932448 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 886/1779] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:20 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:20.936657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e3df5-8fc3-4a37-bf66-a0a236069477 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:20.947707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e3df5-8fc3-4a37-bf66-a0a236069477 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:20.952819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044c530-7693-4b6f-b852-9b9a6f152f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:20.952931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c530-7693-4b6f-b852-9b9a6f152f75 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:20.953196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c530-7693-4b6f-b852-9b9a6f152f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:20.953466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c530-7693-4b6f-b852-9b9a6f152f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:20.954128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044c530-7693-4b6f-b852-9b9a6f152f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:20.955845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044c530-7693-4b6f-b852-9b9a6f152f75 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:20.969215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 894/1780] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:20.970456 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc51205-378f-4c5e-94f5-394941dff495 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:20.980804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc51205-378f-4c5e-94f5-394941dff495 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:20.986479 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e92b3e-ae06-4a3e-a70c-431c04de94b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:20.987085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e92b3e-ae06-4a3e-a70c-431c04de94b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:20.987490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e92b3e-ae06-4a3e-a70c-431c04de94b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:20.987712 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e92b3e-ae06-4a3e-a70c-431c04de94b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:20.988360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e92b3e-ae06-4a3e-a70c-431c04de94b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:20.990061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e92b3e-ae06-4a3e-a70c-431c04de94b4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:21.010441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 887/1781] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:43:21.014706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044c530-7693-4b6f-b852-9b9a6f152f75 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:21.026255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044c530-7693-4b6f-b852-9b9a6f152f75 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:21.030791 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119add17-c9cc-4fdb-ad11-a4a9d92c20ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:21.031468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119add17-c9cc-4fdb-ad11-a4a9d92c20ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:21.031745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119add17-c9cc-4fdb-ad11-a4a9d92c20ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:21.032011 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119add17-c9cc-4fdb-ad11-a4a9d92c20ff admin admin] PATH_INFO: `/v3/projects/ea56664d66944912becf2c179d3a2a54/users/ccc56c17503e42b4adf02c1621f5fbe3/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:21.047276 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119add17-c9cc-4fdb-ad11-a4a9d92c20ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea56664d66944912becf2c179d3a2a54, user_id=ccc56c17503e42b4adf02c1621f5fbe3, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:21.050832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119add17-c9cc-4fdb-ad11-a4a9d92c20ff admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:21.063358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 895/1782] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:21 2022] PUT /identity/v3/projects/ea56664d66944912becf2c179d3a2a54/users/ccc56c17503e42b4adf02c1621f5fbe3/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:21.075449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958db61-b6bb-463b-842a-10bb336eabd8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:21.075766 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958db61-b6bb-463b-842a-10bb336eabd8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:21.076040 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958db61-b6bb-463b-842a-10bb336eabd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:21.103370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7958db61-b6bb-463b-842a-10bb336eabd8 None None] MFA Rules not processed for user `ccc56c17503e42b4adf02c1621f5fbe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:21.104571 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7958db61-b6bb-463b-842a-10bb336eabd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:21.151526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:21.170030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 888/1783] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:21.175163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119add17-c9cc-4fdb-ad11-a4a9d92c20ff admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:21.233146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:21.237482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119add17-c9cc-4fdb-ad11-a4a9d92c20ff admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:21.243184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b9aa2-2e74-49f4-aeec-e237f2d1be58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:21.243795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b9aa2-2e74-49f4-aeec-e237f2d1be58 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:21.244288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b9aa2-2e74-49f4-aeec-e237f2d1be58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:21.244741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b9aa2-2e74-49f4-aeec-e237f2d1be58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:21.245423 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b9aa2-2e74-49f4-aeec-e237f2d1be58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:21.247514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b9aa2-2e74-49f4-aeec-e237f2d1be58 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:21.274156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 896/1784] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:21 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:21.278147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958db61-b6bb-463b-842a-10bb336eabd8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:21.289425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958db61-b6bb-463b-842a-10bb336eabd8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:21.344249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5d227-bbd6-4eb1-8033-5ea1671e289a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:21.345355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5d227-bbd6-4eb1-8033-5ea1671e289a admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:21.345658 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5d227-bbd6-4eb1-8033-5ea1671e289a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:21.345925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5d227-bbd6-4eb1-8033-5ea1671e289a admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:21.346608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5d227-bbd6-4eb1-8033-5ea1671e289a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:21.348546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5d227-bbd6-4eb1-8033-5ea1671e289a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:21.375136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 889/1785] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:21 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:21.376317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b9aa2-2e74-49f4-aeec-e237f2d1be58 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:21.388965 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b9aa2-2e74-49f4-aeec-e237f2d1be58 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:21.393855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb019c8-6e27-45a7-8633-a30bdc8c586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:21.394574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb019c8-6e27-45a7-8633-a30bdc8c586e admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:21.394915 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb019c8-6e27-45a7-8633-a30bdc8c586e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:21.395132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb019c8-6e27-45a7-8633-a30bdc8c586e admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:21.395756 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb019c8-6e27-45a7-8633-a30bdc8c586e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:21.397343 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb019c8-6e27-45a7-8633-a30bdc8c586e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:21.408970 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 897/1786] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:21.413307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5d227-bbd6-4eb1-8033-5ea1671e289a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:21.424376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5d227-bbd6-4eb1-8033-5ea1671e289a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:21.429324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f12f98-d87f-4573-a6bf-3c5c453ef321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:21.430222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f12f98-d87f-4573-a6bf-3c5c453ef321 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:21.430559 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f12f98-d87f-4573-a6bf-3c5c453ef321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:21.430837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f12f98-d87f-4573-a6bf-3c5c453ef321 admin admin] PATH_INFO: `/v3/projects/b436e5c6ae7b488485d919d4ff678e94/users/3fbfd3c6182e4399902b00762d1ec383/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:21.446460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f12f98-d87f-4573-a6bf-3c5c453ef321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b436e5c6ae7b488485d919d4ff678e94, user_id=3fbfd3c6182e4399902b00762d1ec383, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:21.449171 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f12f98-d87f-4573-a6bf-3c5c453ef321 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:21.462911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 890/1787] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:21 2022] PUT /identity/v3/projects/b436e5c6ae7b488485d919d4ff678e94/users/3fbfd3c6182e4399902b00762d1ec383/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:21.466930 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb019c8-6e27-45a7-8633-a30bdc8c586e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:21.477789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb019c8-6e27-45a7-8633-a30bdc8c586e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:21.540780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65ef62-837f-463b-ba51-b61f419796fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:21.541388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65ef62-837f-463b-ba51-b61f419796fc admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:21.541666 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65ef62-837f-463b-ba51-b61f419796fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:21.541937 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65ef62-837f-463b-ba51-b61f419796fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:21.542598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65ef62-837f-463b-ba51-b61f419796fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:21.544290 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a65ef62-837f-463b-ba51-b61f419796fc admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:21.554700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 898/1788] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:21.558856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f12f98-d87f-4573-a6bf-3c5c453ef321 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:21.569453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f12f98-d87f-4573-a6bf-3c5c453ef321 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:21.573969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca35b0-b67e-4110-b391-32929ac91ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:21.574688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca35b0-b67e-4110-b391-32929ac91ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:21.574960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca35b0-b67e-4110-b391-32929ac91ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:21.575225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca35b0-b67e-4110-b391-32929ac91ebe admin admin] PATH_INFO: `/v3/projects/b436e5c6ae7b488485d919d4ff678e94/users/3fbfd3c6182e4399902b00762d1ec383/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:21.577074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca35b0-b67e-4110-b391-32929ac91ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b436e5c6ae7b488485d919d4ff678e94, user_id=3fbfd3c6182e4399902b00762d1ec383, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:21.579986 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca35b0-b67e-4110-b391-32929ac91ebe admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:21.589979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 891/1789] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:21 2022] PUT /identity/v3/projects/b436e5c6ae7b488485d919d4ff678e94/users/3fbfd3c6182e4399902b00762d1ec383/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:21.600524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6779d-1b84-4d09-8c2f-b285f2cf4fd1 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:21.600810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6779d-1b84-4d09-8c2f-b285f2cf4fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:21.601082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6779d-1b84-4d09-8c2f-b285f2cf4fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:21.629874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c6b6779d-1b84-4d09-8c2f-b285f2cf4fd1 None None] MFA Rules not processed for user `3fbfd3c6182e4399902b00762d1ec383`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:21.631170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c6b6779d-1b84-4d09-8c2f-b285f2cf4fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:21.698807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 899/1790] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:21 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:21.710127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca35b0-b67e-4110-b391-32929ac91ebe admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:21.711837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-dcca35b0-b67e-4110-b391-32929ac91ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:21.768205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca35b0-b67e-4110-b391-32929ac91ebe admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:21.771869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead75d66-b7be-4afd-bcf9-f25a66cf3793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:21.772464 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead75d66-b7be-4afd-bcf9-f25a66cf3793 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:21.772744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead75d66-b7be-4afd-bcf9-f25a66cf3793 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:21.773014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead75d66-b7be-4afd-bcf9-f25a66cf3793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:21.781959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead75d66-b7be-4afd-bcf9-f25a66cf3793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:21.784300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead75d66-b7be-4afd-bcf9-f25a66cf3793 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:21.797674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 892/1791] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:21 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:43:22.190299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6779d-1b84-4d09-8c2f-b285f2cf4fd1 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:22.242198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:22.254077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6779d-1b84-4d09-8c2f-b285f2cf4fd1 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:22.258572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:22.259202 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:22.259476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:22.259744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] PATH_INFO: `/v3/users/ccc56c17503e42b4adf02c1621f5fbe3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:22.260840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc56c17503e42b4adf02c1621f5fbe3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:22.262761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:22.309818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc56c17503e42b4adf02c1621f5fbe3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:22.310485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc56c17503e42b4adf02c1621f5fbe3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:22.322615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc56c17503e42b4adf02c1621f5fbe3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:22.323256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc56c17503e42b4adf02c1621f5fbe3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:22.330009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc56c17503e42b4adf02c1621f5fbe3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:22.342767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 900/1792] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:22 2022] DELETE /identity/v3/users/ccc56c17503e42b4adf02c1621f5fbe3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:22.347854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349abd23-0b4e-4254-95da-1352c598c181 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:22.348334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349abd23-0b4e-4254-95da-1352c598c181 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:22.348435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349abd23-0b4e-4254-95da-1352c598c181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:22.367339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-349abd23-0b4e-4254-95da-1352c598c181 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:22.368531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-349abd23-0b4e-4254-95da-1352c598c181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:22.408936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:22.429741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 893/1793] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:22.440556 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:22.442006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:22.499859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2732bf7-6bba-4b00-b804-50ac907d1800 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:22.504643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec291046-6ffe-43ec-963c-fe225d8ed983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:22.505243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec291046-6ffe-43ec-963c-fe225d8ed983 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:22.505515 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec291046-6ffe-43ec-963c-fe225d8ed983 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:22.505782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec291046-6ffe-43ec-963c-fe225d8ed983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:22.514607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec291046-6ffe-43ec-963c-fe225d8ed983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:22.516017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec291046-6ffe-43ec-963c-fe225d8ed983 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:22.525783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 901/1794] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:22.814556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349abd23-0b4e-4254-95da-1352c598c181 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:22.816196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-349abd23-0b4e-4254-95da-1352c598c181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:22.828128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349abd23-0b4e-4254-95da-1352c598c181 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:22.833967 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:22.834660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:22.834938 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:22.835212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] PATH_INFO: `/v3/projects/ea56664d66944912becf2c179d3a2a54` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:22.836385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea56664d66944912becf2c179d3a2a54)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:22.838671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:22.886526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea56664d66944912becf2c179d3a2a54', 'request_id': 'req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:22.894060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea56664d66944912becf2c179d3a2a54', 'request_id': 'req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:22.900742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea56664d66944912becf2c179d3a2a54', 'request_id': 'req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:22.912040 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] The token cache is being invalidate because project ea56664d66944912becf2c179d3a2a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:22.912333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:22.913230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:22.914846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 894/1795] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:22 2022] DELETE /identity/v3/projects/ea56664d66944912becf2c179d3a2a54 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:22.918959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec291046-6ffe-43ec-963c-fe225d8ed983 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:22.929400 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec291046-6ffe-43ec-963c-fe225d8ed983 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:22.991905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73399e2f-5d73-40ff-9fd7-8217a1f10632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:22.992541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73399e2f-5d73-40ff-9fd7-8217a1f10632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:22.992935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73399e2f-5d73-40ff-9fd7-8217a1f10632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:22.993206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73399e2f-5d73-40ff-9fd7-8217a1f10632 admin admin] PATH_INFO: `/v3/users/3fbfd3c6182e4399902b00762d1ec383` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:22.994357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73399e2f-5d73-40ff-9fd7-8217a1f10632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fbfd3c6182e4399902b00762d1ec383)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:22.996447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73399e2f-5d73-40ff-9fd7-8217a1f10632 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:23.012712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:23.044652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-73399e2f-5d73-40ff-9fd7-8217a1f10632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fbfd3c6182e4399902b00762d1ec383'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:23.045308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-73399e2f-5d73-40ff-9fd7-8217a1f10632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fbfd3c6182e4399902b00762d1ec383'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:23.057283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-73399e2f-5d73-40ff-9fd7-8217a1f10632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fbfd3c6182e4399902b00762d1ec383'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:23.057909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-73399e2f-5d73-40ff-9fd7-8217a1f10632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fbfd3c6182e4399902b00762d1ec383'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:23.065200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-73399e2f-5d73-40ff-9fd7-8217a1f10632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fbfd3c6182e4399902b00762d1ec383'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:23.077811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 902/1796] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:22 2022] DELETE /identity/v3/users/3fbfd3c6182e4399902b00762d1ec383 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:23.359095 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:23.361196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:23.374333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8fd07-4649-4e42-b70f-56dde2fb60f7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:23.437579 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:23.438245 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:23.439163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:23.439444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] PATH_INFO: `/v3/projects/b436e5c6ae7b488485d919d4ff678e94` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:23.440559 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b436e5c6ae7b488485d919d4ff678e94)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:23.442477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:23.488247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b436e5c6ae7b488485d919d4ff678e94', 'request_id': 'req-d5c38c59-b22c-4814-be98-4bd9d369f95e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:23.495087 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b436e5c6ae7b488485d919d4ff678e94', 'request_id': 'req-d5c38c59-b22c-4814-be98-4bd9d369f95e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:23.502287 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b436e5c6ae7b488485d919d4ff678e94', 'request_id': 'req-d5c38c59-b22c-4814-be98-4bd9d369f95e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:23.508505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:23.513418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] The token cache is being invalidate because project b436e5c6ae7b488485d919d4ff678e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:23.513885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:23.514949 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d5c38c59-b22c-4814-be98-4bd9d369f95e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:23.517236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 895/1797] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:23 2022] DELETE /identity/v3/projects/b436e5c6ae7b488485d919d4ff678e94 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:23.522265 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3ee9-1f47-49f2-bfb0-45b93fcc472c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:23.522772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3ee9-1f47-49f2-bfb0-45b93fcc472c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:23.522918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f3ee9-1f47-49f2-bfb0-45b93fcc472c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:23.543258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e76f3ee9-1f47-49f2-bfb0-45b93fcc472c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:23.544427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e76f3ee9-1f47-49f2-bfb0-45b93fcc472c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:23.602912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 903/1798] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:23.611165 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767afb2-8e79-440f-bcb8-754a87ad8075 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:23.611453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767afb2-8e79-440f-bcb8-754a87ad8075 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:23.611772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767afb2-8e79-440f-bcb8-754a87ad8075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:23.631139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d767afb2-8e79-440f-bcb8-754a87ad8075 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:23.632326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d767afb2-8e79-440f-bcb8-754a87ad8075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:23.640165 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 896/1799] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:23.644814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f3ee9-1f47-49f2-bfb0-45b93fcc472c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:23.654486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f3ee9-1f47-49f2-bfb0-45b93fcc472c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:23.659809 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641ef80-f704-4690-8361-9854c8c6a7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:23.660396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641ef80-f704-4690-8361-9854c8c6a7be admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:23.660674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641ef80-f704-4690-8361-9854c8c6a7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:23.660941 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641ef80-f704-4690-8361-9854c8c6a7be admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:23.661605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641ef80-f704-4690-8361-9854c8c6a7be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:23.661902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641ef80-f704-4690-8361-9854c8c6a7be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:23.663519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641ef80-f704-4690-8361-9854c8c6a7be admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:23.675859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 904/1800] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:23.680090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767afb2-8e79-440f-bcb8-754a87ad8075 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:23.689260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767afb2-8e79-440f-bcb8-754a87ad8075 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:23.693885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deea0c4-0460-46c4-bbe1-927b4fdc6467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:23.694517 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deea0c4-0460-46c4-bbe1-927b4fdc6467 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:23.694795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deea0c4-0460-46c4-bbe1-927b4fdc6467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:23.695061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deea0c4-0460-46c4-bbe1-927b4fdc6467 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:23.695700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deea0c4-0460-46c4-bbe1-927b4fdc6467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:23.697588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deea0c4-0460-46c4-bbe1-927b4fdc6467 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:23.727917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 897/1801] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:23 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:23.732222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641ef80-f704-4690-8361-9854c8c6a7be admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:23.742913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641ef80-f704-4690-8361-9854c8c6a7be admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:23.800601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42f38e-5010-4ef5-9219-e13d7650e216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:23.801222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42f38e-5010-4ef5-9219-e13d7650e216 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:23.801482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42f38e-5010-4ef5-9219-e13d7650e216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:23.801750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42f38e-5010-4ef5-9219-e13d7650e216 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:23.802445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42f38e-5010-4ef5-9219-e13d7650e216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:23.804332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42f38e-5010-4ef5-9219-e13d7650e216 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:23.834591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 905/1802] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:23 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:23.838732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deea0c4-0460-46c4-bbe1-927b4fdc6467 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:23.849915 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deea0c4-0460-46c4-bbe1-927b4fdc6467 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:23.854632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a47ae-dea2-481e-8b88-4f066ed5c8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:23.855230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a47ae-dea2-481e-8b88-4f066ed5c8ab admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:23.855503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a47ae-dea2-481e-8b88-4f066ed5c8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:23.855773 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a47ae-dea2-481e-8b88-4f066ed5c8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:23.856407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a47ae-dea2-481e-8b88-4f066ed5c8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:23.858119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a47ae-dea2-481e-8b88-4f066ed5c8ab admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:23.870616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 898/1803] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:23.873051 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42f38e-5010-4ef5-9219-e13d7650e216 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:23.883935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42f38e-5010-4ef5-9219-e13d7650e216 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:23.888856 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c381a-8c2e-48e0-83d0-3487e68326ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:23.889453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c381a-8c2e-48e0-83d0-3487e68326ce admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:23.889727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c381a-8c2e-48e0-83d0-3487e68326ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:23.890001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c381a-8c2e-48e0-83d0-3487e68326ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:23.890736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c381a-8c2e-48e0-83d0-3487e68326ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:23.892943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c381a-8c2e-48e0-83d0-3487e68326ce admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:23.898353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:23.914870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 906/1804] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:43:23.916180 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a47ae-dea2-481e-8b88-4f066ed5c8ab admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:23.927794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a47ae-dea2-481e-8b88-4f066ed5c8ab admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:23.932373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c04f75-4dbe-4877-b1de-f0a8c0189dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:23.933017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c04f75-4dbe-4877-b1de-f0a8c0189dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:23.933293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c04f75-4dbe-4877-b1de-f0a8c0189dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:23.933569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c04f75-4dbe-4877-b1de-f0a8c0189dcc admin admin] PATH_INFO: `/v3/projects/0872569f54264498a16218bbb291ae9e/users/54a74bcb89c24ae99df8adf6e1106060/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:23.948376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c04f75-4dbe-4877-b1de-f0a8c0189dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0872569f54264498a16218bbb291ae9e, user_id=54a74bcb89c24ae99df8adf6e1106060, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:23.951047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c04f75-4dbe-4877-b1de-f0a8c0189dcc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:23.964098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 899/1805] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:23 2022] PUT /identity/v3/projects/0872569f54264498a16218bbb291ae9e/users/54a74bcb89c24ae99df8adf6e1106060/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:23.977101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa9f6a-3b64-4682-bde1-e6c0faa82a5b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:23.978140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa9f6a-3b64-4682-bde1-e6c0faa82a5b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:23.978140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa9f6a-3b64-4682-bde1-e6c0faa82a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:24.010218 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c1fa9f6a-3b64-4682-bde1-e6c0faa82a5b None None] MFA Rules not processed for user `54a74bcb89c24ae99df8adf6e1106060`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:24.011414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c1fa9f6a-3b64-4682-bde1-e6c0faa82a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:24.079594 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 907/1806] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:24.084127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c04f75-4dbe-4877-b1de-f0a8c0189dcc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:24.146792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c04f75-4dbe-4877-b1de-f0a8c0189dcc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:24.151370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804215f-843c-4b19-b165-459e0f4972dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:24.151973 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804215f-843c-4b19-b165-459e0f4972dc admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:24.152252 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804215f-843c-4b19-b165-459e0f4972dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:24.152524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804215f-843c-4b19-b165-459e0f4972dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:24.153151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804215f-843c-4b19-b165-459e0f4972dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:24.155128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804215f-843c-4b19-b165-459e0f4972dc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:24.193415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 900/1807] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:24 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:24.194840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa9f6a-3b64-4682-bde1-e6c0faa82a5b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:24.209770 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa9f6a-3b64-4682-bde1-e6c0faa82a5b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:24.275704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384ff25-9d0d-4b38-a930-01ff39e91638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:24.276948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384ff25-9d0d-4b38-a930-01ff39e91638 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:24.277229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384ff25-9d0d-4b38-a930-01ff39e91638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:24.277526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384ff25-9d0d-4b38-a930-01ff39e91638 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:24.278225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1384ff25-9d0d-4b38-a930-01ff39e91638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:24.280149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1384ff25-9d0d-4b38-a930-01ff39e91638 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:24.306366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 908/1808] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:24 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:24.307672 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804215f-843c-4b19-b165-459e0f4972dc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:24.318953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804215f-843c-4b19-b165-459e0f4972dc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:24.323676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660499d-e964-4476-8406-79e8875edebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:24.324335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660499d-e964-4476-8406-79e8875edebc admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:24.324681 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660499d-e964-4476-8406-79e8875edebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:24.325021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660499d-e964-4476-8406-79e8875edebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:24.325657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660499d-e964-4476-8406-79e8875edebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:24.327337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660499d-e964-4476-8406-79e8875edebc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:24.339160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 901/1809] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:24.343411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384ff25-9d0d-4b38-a930-01ff39e91638 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:24.354062 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384ff25-9d0d-4b38-a930-01ff39e91638 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:24.359817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004c720-2f86-4501-bb35-f58675330f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:24.361051 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004c720-2f86-4501-bb35-f58675330f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:24.361355 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004c720-2f86-4501-bb35-f58675330f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:24.361673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004c720-2f86-4501-bb35-f58675330f6b admin admin] PATH_INFO: `/v3/projects/76bdb387f35e48aa81dc65a873cf8179/users/c0868856481c471586d91ddf103acd1b/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:24.382121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004c720-2f86-4501-bb35-f58675330f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=76bdb387f35e48aa81dc65a873cf8179, user_id=c0868856481c471586d91ddf103acd1b, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:24.384723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004c720-2f86-4501-bb35-f58675330f6b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:24.400920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 909/1810] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:24 2022] PUT /identity/v3/projects/76bdb387f35e48aa81dc65a873cf8179/users/c0868856481c471586d91ddf103acd1b/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:24.404917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660499d-e964-4476-8406-79e8875edebc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:24.416368 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660499d-e964-4476-8406-79e8875edebc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:24.475284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df921c-2394-4f0e-aa2e-049f882cb498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:24.475892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df921c-2394-4f0e-aa2e-049f882cb498 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:24.476169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df921c-2394-4f0e-aa2e-049f882cb498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:24.476469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df921c-2394-4f0e-aa2e-049f882cb498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:24.477108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df921c-2394-4f0e-aa2e-049f882cb498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:24.478727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df921c-2394-4f0e-aa2e-049f882cb498 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:24.489350 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 902/1811] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:43:24.493287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004c720-2f86-4501-bb35-f58675330f6b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:24.505066 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004c720-2f86-4501-bb35-f58675330f6b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:24.509996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496be80-528f-48fa-98b5-74b00c2f6c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:24.510494 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496be80-528f-48fa-98b5-74b00c2f6c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:24.511167 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496be80-528f-48fa-98b5-74b00c2f6c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:24.511535 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496be80-528f-48fa-98b5-74b00c2f6c8c admin admin] PATH_INFO: `/v3/projects/76bdb387f35e48aa81dc65a873cf8179/users/c0868856481c471586d91ddf103acd1b/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:24.513893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496be80-528f-48fa-98b5-74b00c2f6c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=76bdb387f35e48aa81dc65a873cf8179, user_id=c0868856481c471586d91ddf103acd1b, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:24.517625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496be80-528f-48fa-98b5-74b00c2f6c8c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:24.529915 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 910/1812] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:24 2022] PUT /identity/v3/projects/76bdb387f35e48aa81dc65a873cf8179/users/c0868856481c471586d91ddf103acd1b/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:24.540691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd88e46-dea4-4d3f-a4e4-a08044c4664a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:24.541013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd88e46-dea4-4d3f-a4e4-a08044c4664a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:24.541290 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd88e46-dea4-4d3f-a4e4-a08044c4664a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:24.549434 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:24.571382 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4bd88e46-dea4-4d3f-a4e4-a08044c4664a None None] MFA Rules not processed for user `c0868856481c471586d91ddf103acd1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:24.572630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4bd88e46-dea4-4d3f-a4e4-a08044c4664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:24.641054 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 903/1813] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:24.653960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496be80-528f-48fa-98b5-74b00c2f6c8c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:24.655866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a496be80-528f-48fa-98b5-74b00c2f6c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:24.704435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:24.716604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496be80-528f-48fa-98b5-74b00c2f6c8c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:24.720505 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da0b66-e5f1-4388-ac32-c2a2e058e510 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:24.721112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da0b66-e5f1-4388-ac32-c2a2e058e510 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:24.721389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da0b66-e5f1-4388-ac32-c2a2e058e510 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:24.721659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da0b66-e5f1-4388-ac32-c2a2e058e510 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:24.730499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da0b66-e5f1-4388-ac32-c2a2e058e510 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:24.731922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da0b66-e5f1-4388-ac32-c2a2e058e510 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:24.741888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 911/1814] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:43:24.812565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd88e46-dea4-4d3f-a4e4-a08044c4664a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:24.822876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd88e46-dea4-4d3f-a4e4-a08044c4664a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:24.826472 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38979296-1c2a-45b9-8199-d92e073b5480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:24.827069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38979296-1c2a-45b9-8199-d92e073b5480 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:24.827374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38979296-1c2a-45b9-8199-d92e073b5480 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:24.827647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38979296-1c2a-45b9-8199-d92e073b5480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:24.883440 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38979296-1c2a-45b9-8199-d92e073b5480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:24.885003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38979296-1c2a-45b9-8199-d92e073b5480 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:24.893913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 904/1815] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:25.139579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4996c-3f08-45ab-bbb6-869bf585a773 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:25.139863 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4996c-3f08-45ab-bbb6-869bf585a773 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:25.140136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4996c-3f08-45ab-bbb6-869bf585a773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:25.159301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a7c4996c-3f08-45ab-bbb6-869bf585a773 None None] MFA Rules not processed for user `687126fda40f481cb5ba5d185cbfa74a`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:25.160497 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a7c4996c-3f08-45ab-bbb6-869bf585a773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:25.191977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38979296-1c2a-45b9-8199-d92e073b5480 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:25.231769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 912/1816] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:25.270844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38979296-1c2a-45b9-8199-d92e073b5480 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:25.275379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:25.276116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:25.276464 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:25.276739 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] PATH_INFO: `/v3/users/54a74bcb89c24ae99df8adf6e1106060` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:25.277883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a74bcb89c24ae99df8adf6e1106060)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:25.279819 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:25.316748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:25.332917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a74bcb89c24ae99df8adf6e1106060'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:25.333562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a74bcb89c24ae99df8adf6e1106060'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:25.334202 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a74bcb89c24ae99df8adf6e1106060'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:25.341160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a74bcb89c24ae99df8adf6e1106060'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:25.353664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a74bcb89c24ae99df8adf6e1106060'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:25.367555 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 905/1817] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:25 2022] DELETE /identity/v3/users/54a74bcb89c24ae99df8adf6e1106060 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:25.372167 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdd177-173e-4823-a2b7-d4ee088aabe9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:25.372473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdd177-173e-4823-a2b7-d4ee088aabe9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:25.372742 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdd177-173e-4823-a2b7-d4ee088aabe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:25.393162 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-60cdd177-173e-4823-a2b7-d4ee088aabe9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:25.394452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-60cdd177-173e-4823-a2b7-d4ee088aabe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:25.461882 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 913/1818] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:25.472567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:25.474041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:25.529987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1383aa4-15e0-4546-8f41-bd16bdc75d73 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:25.534784 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2954d6-71ec-4560-83a3-80102e84fe91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:25.535383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2954d6-71ec-4560-83a3-80102e84fe91 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:25.535661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2954d6-71ec-4560-83a3-80102e84fe91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:25.535959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2954d6-71ec-4560-83a3-80102e84fe91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:25.544943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2954d6-71ec-4560-83a3-80102e84fe91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:25.546384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2954d6-71ec-4560-83a3-80102e84fe91 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:25.559189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 906/1819] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:25.855477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdd177-173e-4823-a2b7-d4ee088aabe9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:25.856904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-60cdd177-173e-4823-a2b7-d4ee088aabe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:25.870615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdd177-173e-4823-a2b7-d4ee088aabe9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:25.878115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:25.878785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:25.879064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:25.879328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] PATH_INFO: `/v3/projects/0872569f54264498a16218bbb291ae9e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:25.880882 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0872569f54264498a16218bbb291ae9e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:25.882889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:25.936344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0872569f54264498a16218bbb291ae9e', 'request_id': 'req-d0a59666-056d-4cb4-9c4f-1ae2b860820d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:25.942050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0872569f54264498a16218bbb291ae9e', 'request_id': 'req-d0a59666-056d-4cb4-9c4f-1ae2b860820d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:25.951047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0872569f54264498a16218bbb291ae9e', 'request_id': 'req-d0a59666-056d-4cb4-9c4f-1ae2b860820d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:25.962769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] The token cache is being invalidate because project 0872569f54264498a16218bbb291ae9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:25.963052 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:25.963660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:25.965311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 914/1820] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:25 2022] DELETE /identity/v3/projects/0872569f54264498a16218bbb291ae9e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:25.969074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2954d6-71ec-4560-83a3-80102e84fe91 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:25.980607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2954d6-71ec-4560-83a3-80102e84fe91 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:26.043213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e6bb3-387f-44c4-bce6-59bfdff11fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:26.043853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e6bb3-387f-44c4-bce6-59bfdff11fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:26.044131 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e6bb3-387f-44c4-bce6-59bfdff11fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:26.044401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e6bb3-387f-44c4-bce6-59bfdff11fc6 admin admin] PATH_INFO: `/v3/users/c0868856481c471586d91ddf103acd1b` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:26.045514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e6bb3-387f-44c4-bce6-59bfdff11fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0868856481c471586d91ddf103acd1b)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:26.047460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e6bb3-387f-44c4-bce6-59bfdff11fc6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:26.099755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-070e6bb3-387f-44c4-bce6-59bfdff11fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0868856481c471586d91ddf103acd1b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:26.100624 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-070e6bb3-387f-44c4-bce6-59bfdff11fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0868856481c471586d91ddf103acd1b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:26.101406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-070e6bb3-387f-44c4-bce6-59bfdff11fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0868856481c471586d91ddf103acd1b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:26.107108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:26.112585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-070e6bb3-387f-44c4-bce6-59bfdff11fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0868856481c471586d91ddf103acd1b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:26.148896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-070e6bb3-387f-44c4-bce6-59bfdff11fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0868856481c471586d91ddf103acd1b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:26.166470 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 907/1821] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:25 2022] DELETE /identity/v3/users/c0868856481c471586d91ddf103acd1b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:26.459708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:26.461374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:26.480322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a59666-056d-4cb4-9c4f-1ae2b860820d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:26.497321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:26.547310 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:26.548578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:26.548856 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:26.549127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] PATH_INFO: `/v3/projects/76bdb387f35e48aa81dc65a873cf8179` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:26.550283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76bdb387f35e48aa81dc65a873cf8179)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:26.552169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:26.597507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76bdb387f35e48aa81dc65a873cf8179', 'request_id': 'req-7b23cec1-4210-48f7-bcc9-1e09e62fab71'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:26.602960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76bdb387f35e48aa81dc65a873cf8179', 'request_id': 'req-7b23cec1-4210-48f7-bcc9-1e09e62fab71'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:26.610139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76bdb387f35e48aa81dc65a873cf8179', 'request_id': 'req-7b23cec1-4210-48f7-bcc9-1e09e62fab71'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:26.621769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] The token cache is being invalidate because project 76bdb387f35e48aa81dc65a873cf8179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:26.622055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:26.622702 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7b23cec1-4210-48f7-bcc9-1e09e62fab71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:26.624154 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 915/1822] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:26 2022] DELETE /identity/v3/projects/76bdb387f35e48aa81dc65a873cf8179 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:26.629700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49e4ac-67ff-4291-a834-05269fba46bb None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:26.630215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49e4ac-67ff-4291-a834-05269fba46bb None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:26.630309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49e4ac-67ff-4291-a834-05269fba46bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:26.649539 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9b49e4ac-67ff-4291-a834-05269fba46bb None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:26.650752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9b49e4ac-67ff-4291-a834-05269fba46bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:26.708772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 908/1823] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:26.717310 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df7c7-ddd0-4b04-855e-4577c83d467b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:26.717687 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df7c7-ddd0-4b04-855e-4577c83d467b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:26.717970 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df7c7-ddd0-4b04-855e-4577c83d467b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:26.739040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-6a5df7c7-ddd0-4b04-855e-4577c83d467b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:26.740284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6a5df7c7-ddd0-4b04-855e-4577c83d467b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:26.748204 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 916/1824] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:26.752444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49e4ac-67ff-4291-a834-05269fba46bb None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:26.762149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49e4ac-67ff-4291-a834-05269fba46bb None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:26.766787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e0f33-3ed5-4306-b390-f39c56f13081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:26.767378 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e0f33-3ed5-4306-b390-f39c56f13081 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:26.767653 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e0f33-3ed5-4306-b390-f39c56f13081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:26.767924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e0f33-3ed5-4306-b390-f39c56f13081 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:26.768564 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e0f33-3ed5-4306-b390-f39c56f13081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:26.768865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e0f33-3ed5-4306-b390-f39c56f13081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:26.770538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e0f33-3ed5-4306-b390-f39c56f13081 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:26.781748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 909/1825] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:26.786740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5df7c7-ddd0-4b04-855e-4577c83d467b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:26.795695 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5df7c7-ddd0-4b04-855e-4577c83d467b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:26.801544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98b107-d424-4ab2-a330-861adae3cfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:26.802136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98b107-d424-4ab2-a330-861adae3cfd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:26.802445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98b107-d424-4ab2-a330-861adae3cfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:26.802715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98b107-d424-4ab2-a330-861adae3cfd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:26.803351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98b107-d424-4ab2-a330-861adae3cfd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:26.805263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98b107-d424-4ab2-a330-861adae3cfd0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:26.832213 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 917/1826] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:26 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:26.836298 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e0f33-3ed5-4306-b390-f39c56f13081 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:26.846803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e0f33-3ed5-4306-b390-f39c56f13081 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:26.904933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30346aec-bec5-4901-ae5d-e0b32f2e0728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:26.905545 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30346aec-bec5-4901-ae5d-e0b32f2e0728 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:26.905826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30346aec-bec5-4901-ae5d-e0b32f2e0728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:26.906096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30346aec-bec5-4901-ae5d-e0b32f2e0728 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:26.906801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30346aec-bec5-4901-ae5d-e0b32f2e0728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:26.908979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30346aec-bec5-4901-ae5d-e0b32f2e0728 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:26.935435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 910/1827] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:26 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:26.936741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98b107-d424-4ab2-a330-861adae3cfd0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:26.947847 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98b107-d424-4ab2-a330-861adae3cfd0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:26.956053 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72113d-9f64-4d36-b8b7-c7ef8c729268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:26.956660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72113d-9f64-4d36-b8b7-c7ef8c729268 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:26.956941 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72113d-9f64-4d36-b8b7-c7ef8c729268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:26.957212 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72113d-9f64-4d36-b8b7-c7ef8c729268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:26.957849 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72113d-9f64-4d36-b8b7-c7ef8c729268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:26.959522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72113d-9f64-4d36-b8b7-c7ef8c729268 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:26.969459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 918/1828] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:26.973451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30346aec-bec5-4901-ae5d-e0b32f2e0728 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:26.983651 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30346aec-bec5-4901-ae5d-e0b32f2e0728 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:26.988925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716ca76-d453-4c96-9d2b-9ebdd6200055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:26.989519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716ca76-d453-4c96-9d2b-9ebdd6200055 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:26.989795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716ca76-d453-4c96-9d2b-9ebdd6200055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:26.990094 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716ca76-d453-4c96-9d2b-9ebdd6200055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:26.990768 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716ca76-d453-4c96-9d2b-9ebdd6200055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:26.992345 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716ca76-d453-4c96-9d2b-9ebdd6200055 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:27.003495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 911/1829] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:27.007495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72113d-9f64-4d36-b8b7-c7ef8c729268 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:27.017820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72113d-9f64-4d36-b8b7-c7ef8c729268 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:27.022484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca302357-7e8a-4dad-9b1f-eee18c933545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:27.023128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca302357-7e8a-4dad-9b1f-eee18c933545 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:27.023406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca302357-7e8a-4dad-9b1f-eee18c933545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:27.023678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca302357-7e8a-4dad-9b1f-eee18c933545 admin admin] PATH_INFO: `/v3/projects/7930734bac2c4182bc42e6ce3112c646/users/083c3a810470453f831183843ba03707/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:27.038712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca302357-7e8a-4dad-9b1f-eee18c933545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7930734bac2c4182bc42e6ce3112c646, user_id=083c3a810470453f831183843ba03707, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:27.041348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca302357-7e8a-4dad-9b1f-eee18c933545 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:27.054291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 919/1830] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:27 2022] PUT /identity/v3/projects/7930734bac2c4182bc42e6ce3112c646/users/083c3a810470453f831183843ba03707/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:27.065104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2844ca-657f-475e-9310-e38fa30e298b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:27.065417 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2844ca-657f-475e-9310-e38fa30e298b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:27.065690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2844ca-657f-475e-9310-e38fa30e298b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:27.094400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-cd2844ca-657f-475e-9310-e38fa30e298b None None] MFA Rules not processed for user `083c3a810470453f831183843ba03707`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:27.095919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cd2844ca-657f-475e-9310-e38fa30e298b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:27.124700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:27.173552 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 912/1831] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:27.184928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca302357-7e8a-4dad-9b1f-eee18c933545 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:27.187006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ca302357-7e8a-4dad-9b1f-eee18c933545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:27.236800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:27.272999 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca302357-7e8a-4dad-9b1f-eee18c933545 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:27.277168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514ed22-d583-423e-b2f1-989fed068633 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07f8443d5594f47bc81e9f9dc9ce43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:27.277771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514ed22-d583-423e-b2f1-989fed068633 service glance] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:27.278047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514ed22-d583-423e-b2f1-989fed068633 service glance] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:27.278349 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514ed22-d583-423e-b2f1-989fed068633 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:27.291355 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514ed22-d583-423e-b2f1-989fed068633 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:27.292798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514ed22-d583-423e-b2f1-989fed068633 service glance] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:27.302278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 920/1832] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:34.512516 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2844ca-657f-475e-9310-e38fa30e298b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:34.514121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cd2844ca-657f-475e-9310-e38fa30e298b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:34.597331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2844ca-657f-475e-9310-e38fa30e298b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:34.602942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:34.604432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:34.604432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:34.604432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] PATH_INFO: `/v3/users/6e5a93877d764a8295870cd71033cd24` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:34.606068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e5a93877d764a8295870cd71033cd24)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:34.608275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:34.696542 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e5a93877d764a8295870cd71033cd24'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:34.697499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e5a93877d764a8295870cd71033cd24'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:34.699370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e5a93877d764a8295870cd71033cd24'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:34.717551 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e5a93877d764a8295870cd71033cd24'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:34.738999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e5a93877d764a8295870cd71033cd24'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:34.765104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 913/1833] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:34 2022] DELETE /identity/v3/users/6e5a93877d764a8295870cd71033cd24 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:34.770486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714340-983a-4295-bf8e-d6193d569629 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:34.771126 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714340-983a-4295-bf8e-d6193d569629 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:34.771126 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714340-983a-4295-bf8e-d6193d569629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:34.800413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5b714340-983a-4295-bf8e-d6193d569629 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:34.803657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5b714340-983a-4295-bf8e-d6193d569629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:34.905534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 921/1834] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:34.916895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:34.920221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:34.979765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:34.993236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7335d1f-5f3b-42ec-aabb-9dc0b3b5f57c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:34.997261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd7dc5-eb3a-4a0d-a4f8-f375c139d09c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:34.997875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd7dc5-eb3a-4a0d-a4f8-f375c139d09c service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:34.998152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd7dc5-eb3a-4a0d-a4f8-f375c139d09c service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:34.998447 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd7dc5-eb3a-4a0d-a4f8-f375c139d09c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:35.012646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd7dc5-eb3a-4a0d-a4f8-f375c139d09c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:35.014114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd7dc5-eb3a-4a0d-a4f8-f375c139d09c service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:35.024622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 914/1835] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:35.333381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b714340-983a-4295-bf8e-d6193d569629 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:35.334893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5b714340-983a-4295-bf8e-d6193d569629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:35.355381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b714340-983a-4295-bf8e-d6193d569629 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:35.360092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:35.360753 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:35.361525 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:35.361797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] PATH_INFO: `/v3/projects/be6ef569c3894d2182483d22eab93189` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:35.363009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be6ef569c3894d2182483d22eab93189)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:35.365073 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:35.432707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be6ef569c3894d2182483d22eab93189', 'request_id': 'req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:35.443067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be6ef569c3894d2182483d22eab93189', 'request_id': 'req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:35.452606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be6ef569c3894d2182483d22eab93189', 'request_id': 'req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:35.481749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] The token cache is being invalidate because project be6ef569c3894d2182483d22eab93189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:35.482044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:35.482732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c02800c5-e5d7-49a1-9ea4-5cbddd701d73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:35.484413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 922/1836] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:35 2022] DELETE /identity/v3/projects/be6ef569c3894d2182483d22eab93189 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:35.489994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef012473-d5b6-40c1-970b-5e193e721aad None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:35.490584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef012473-d5b6-40c1-970b-5e193e721aad None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:35.490930 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef012473-d5b6-40c1-970b-5e193e721aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:35.517846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ef012473-d5b6-40c1-970b-5e193e721aad None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:35.519092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ef012473-d5b6-40c1-970b-5e193e721aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:35.640248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 915/1837] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:35.651010 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4d63f-2840-4c15-b1b6-e432e4c10bc4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:35.651933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4d63f-2840-4c15-b1b6-e432e4c10bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:35.652210 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4d63f-2840-4c15-b1b6-e432e4c10bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:35.722072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-76b4d63f-2840-4c15-b1b6-e432e4c10bc4 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:35.724257 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-76b4d63f-2840-4c15-b1b6-e432e4c10bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:35.738205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 923/1838] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:35.743264 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef012473-d5b6-40c1-970b-5e193e721aad None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:35.758743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef012473-d5b6-40c1-970b-5e193e721aad None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:35.766198 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf74dab-1ed2-4d53-877f-5143257bf5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:35.766825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf74dab-1ed2-4d53-877f-5143257bf5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:35.767104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf74dab-1ed2-4d53-877f-5143257bf5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:35.769110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf74dab-1ed2-4d53-877f-5143257bf5a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:35.769495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf74dab-1ed2-4d53-877f-5143257bf5a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:35.769752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf74dab-1ed2-4d53-877f-5143257bf5a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:35.771400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf74dab-1ed2-4d53-877f-5143257bf5a7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:35.791966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 916/1839] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:35.796140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4d63f-2840-4c15-b1b6-e432e4c10bc4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:35.807613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4d63f-2840-4c15-b1b6-e432e4c10bc4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:35.819316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4eb4b5-1247-4508-a8e6-086b1da76328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:35.820031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4eb4b5-1247-4508-a8e6-086b1da76328 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:35.820370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4eb4b5-1247-4508-a8e6-086b1da76328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:35.820921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4eb4b5-1247-4508-a8e6-086b1da76328 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:35.821707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4eb4b5-1247-4508-a8e6-086b1da76328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:35.823974 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4eb4b5-1247-4508-a8e6-086b1da76328 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:35.859853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 924/1840] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:35 2022] POST /identity/v3/projects => generated 395 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:35.864548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf74dab-1ed2-4d53-877f-5143257bf5a7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:35.880083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf74dab-1ed2-4d53-877f-5143257bf5a7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:36.000622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a8a7-4d37-4294-b0b3-9b44c6256f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:36.001089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a8a7-4d37-4294-b0b3-9b44c6256f71 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:36.001508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a8a7-4d37-4294-b0b3-9b44c6256f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:36.003261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a8a7-4d37-4294-b0b3-9b44c6256f71 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:36.003856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514a8a7-4d37-4294-b0b3-9b44c6256f71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:36.006107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514a8a7-4d37-4294-b0b3-9b44c6256f71 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:36.043335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 917/1841] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:35 2022] POST /identity/v3/users => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:36.047197 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4eb4b5-1247-4508-a8e6-086b1da76328 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:36.062724 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4eb4b5-1247-4508-a8e6-086b1da76328 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:36.069330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d89670-e053-4368-ad76-120a5fea3658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:36.069330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d89670-e053-4368-ad76-120a5fea3658 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:36.069330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d89670-e053-4368-ad76-120a5fea3658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:36.071723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d89670-e053-4368-ad76-120a5fea3658 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:36.071723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d89670-e053-4368-ad76-120a5fea3658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:36.072422 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d89670-e053-4368-ad76-120a5fea3658 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:36.096021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 925/1842] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:36.099966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a8a7-4d37-4294-b0b3-9b44c6256f71 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:36.114946 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a8a7-4d37-4294-b0b3-9b44c6256f71 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:36.120917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f9143-d3af-43a7-85b8-3d35d5533514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:36.122000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f9143-d3af-43a7-85b8-3d35d5533514 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:36.122633 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f9143-d3af-43a7-85b8-3d35d5533514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:36.123422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f9143-d3af-43a7-85b8-3d35d5533514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:36.124296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f9143-d3af-43a7-85b8-3d35d5533514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:36.128363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f9143-d3af-43a7-85b8-3d35d5533514 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:36.141437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 918/1843] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:36.145967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d89670-e053-4368-ad76-120a5fea3658 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:36.158265 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d89670-e053-4368-ad76-120a5fea3658 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:36.164548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17607048-502b-4ef3-a99c-f2d9bb561a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:36.165261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17607048-502b-4ef3-a99c-f2d9bb561a9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:36.165546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17607048-502b-4ef3-a99c-f2d9bb561a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:36.165813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17607048-502b-4ef3-a99c-f2d9bb561a9a admin admin] PATH_INFO: `/v3/projects/3a23d7348eb742348f46e7a802559cc7/users/528e019cd07148f2b85566c90b1f1be9/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:36.168489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:36.188573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17607048-502b-4ef3-a99c-f2d9bb561a9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a23d7348eb742348f46e7a802559cc7, user_id=528e019cd07148f2b85566c90b1f1be9, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:36.198673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17607048-502b-4ef3-a99c-f2d9bb561a9a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:36.229649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 926/1844] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:36 2022] PUT /identity/v3/projects/3a23d7348eb742348f46e7a802559cc7/users/528e019cd07148f2b85566c90b1f1be9/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:36.252504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89089ad5-ac49-4b4f-b759-5a2d6f1a400b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:36.252504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89089ad5-ac49-4b4f-b759-5a2d6f1a400b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:36.254702 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89089ad5-ac49-4b4f-b759-5a2d6f1a400b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:36.296532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-89089ad5-ac49-4b4f-b759-5a2d6f1a400b None None] MFA Rules not processed for user `528e019cd07148f2b85566c90b1f1be9`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:36.297967 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-89089ad5-ac49-4b4f-b759-5a2d6f1a400b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:36.375484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:36.400486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 919/1845] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:36 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:36.413483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17607048-502b-4ef3-a99c-f2d9bb561a9a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:36.415328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-17607048-502b-4ef3-a99c-f2d9bb561a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:36.495115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17607048-502b-4ef3-a99c-f2d9bb561a9a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:36.503236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627d07c-ba14-4e07-a545-ab5034483425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:36.503866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627d07c-ba14-4e07-a545-ab5034483425 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:36.504167 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627d07c-ba14-4e07-a545-ab5034483425 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:36.504434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627d07c-ba14-4e07-a545-ab5034483425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:36.513823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627d07c-ba14-4e07-a545-ab5034483425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:36.515281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627d07c-ba14-4e07-a545-ab5034483425 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:36.524649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 927/1846] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:36 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:41.686851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85761fec-07ab-48a7-af2d-afdf97759e89 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:41.687374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85761fec-07ab-48a7-af2d-afdf97759e89 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:41.687374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85761fec-07ab-48a7-af2d-afdf97759e89 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:41.688752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 920/1847] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:43:41.701520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8b052-3ce0-4ebf-b9a8-eb1b41630c72 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:41.702057 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8b052-3ce0-4ebf-b9a8-eb1b41630c72 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:41.702057 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8b052-3ce0-4ebf-b9a8-eb1b41630c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:41.722251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-95d8b052-3ce0-4ebf-b9a8-eb1b41630c72 None None] MFA Rules not processed for user `e0ce1753e22d4952ab5d4348b0cffc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:41.723493 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-95d8b052-3ce0-4ebf-b9a8-eb1b41630c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:41.769587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85761fec-07ab-48a7-af2d-afdf97759e89 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:41.771439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-85761fec-07ab-48a7-af2d-afdf97759e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:41.800895 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 928/1848] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:43:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:41.813034 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8b052-3ce0-4ebf-b9a8-eb1b41630c72 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:41.814594 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-95d8b052-3ce0-4ebf-b9a8-eb1b41630c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:41.827396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8b052-3ce0-4ebf-b9a8-eb1b41630c72 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:41.833244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3fb5f-2c19-415e-9e72-74ab526314cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:41.833862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3fb5f-2c19-415e-9e72-74ab526314cd service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:41.834140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3fb5f-2c19-415e-9e72-74ab526314cd service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:41.834439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3fb5f-2c19-415e-9e72-74ab526314cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:41.843616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3fb5f-2c19-415e-9e72-74ab526314cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:41.846075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3fb5f-2c19-415e-9e72-74ab526314cd service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:41.851994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85761fec-07ab-48a7-af2d-afdf97759e89 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:41.856565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100004ed-9cd3-49df-9546-c7b90092b480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:41.856707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 929/1849] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:43:41.857221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100004ed-9cd3-49df-9546-c7b90092b480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:41.857495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100004ed-9cd3-49df-9546-c7b90092b480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:41.857764 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100004ed-9cd3-49df-9546-c7b90092b480 admin admin] PATH_INFO: `/v3/users/9cfcee4879df4843867f69c5091df370` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:41.858919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100004ed-9cd3-49df-9546-c7b90092b480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cfcee4879df4843867f69c5091df370)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:41.860835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100004ed-9cd3-49df-9546-c7b90092b480 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:41.916248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-100004ed-9cd3-49df-9546-c7b90092b480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cfcee4879df4843867f69c5091df370'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:41.917016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-100004ed-9cd3-49df-9546-c7b90092b480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cfcee4879df4843867f69c5091df370'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:41.917642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-100004ed-9cd3-49df-9546-c7b90092b480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cfcee4879df4843867f69c5091df370'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:41.926001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-100004ed-9cd3-49df-9546-c7b90092b480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cfcee4879df4843867f69c5091df370'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:41.937861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-100004ed-9cd3-49df-9546-c7b90092b480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cfcee4879df4843867f69c5091df370'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:41.951297 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 921/1850] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:41 2022] DELETE /identity/v3/users/9cfcee4879df4843867f69c5091df370 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:42.127139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3fb5f-2c19-415e-9e72-74ab526314cd service placement] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:42.138472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3fb5f-2c19-415e-9e72-74ab526314cd service placement] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:42.252051 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:42.252954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:42.253241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:42.253516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] PATH_INFO: `/v3/projects/ab3a235b26aa4c93b2354d2af757873e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:42.254711 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab3a235b26aa4c93b2354d2af757873e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:42.256622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:42.267573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:42.316510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab3a235b26aa4c93b2354d2af757873e', 'request_id': 'req-86f29fef-2ad6-4a97-bce1-c1ea4299af55'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:42.323091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab3a235b26aa4c93b2354d2af757873e', 'request_id': 'req-86f29fef-2ad6-4a97-bce1-c1ea4299af55'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:42.342894 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab3a235b26aa4c93b2354d2af757873e', 'request_id': 'req-86f29fef-2ad6-4a97-bce1-c1ea4299af55'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:42.355811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] The token cache is being invalidate because project ab3a235b26aa4c93b2354d2af757873e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:42.356114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:42.356755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-86f29fef-2ad6-4a97-bce1-c1ea4299af55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:42.362814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 930/1851] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:42 2022] DELETE /identity/v3/projects/ab3a235b26aa4c93b2354d2af757873e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:43:42.367854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9d2f4-0e9c-4b9b-964f-649efa5ff0ea None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:42.368166 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9d2f4-0e9c-4b9b-964f-649efa5ff0ea None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:42.368450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9d2f4-0e9c-4b9b-964f-649efa5ff0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:42.388748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-efc9d2f4-0e9c-4b9b-964f-649efa5ff0ea None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:42.390075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-efc9d2f4-0e9c-4b9b-964f-649efa5ff0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:42.454282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 922/1852] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:42.462520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23fe8-6238-49e4-8259-7926f28de9ff None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:42.462788 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23fe8-6238-49e4-8259-7926f28de9ff None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:42.463065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23fe8-6238-49e4-8259-7926f28de9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:42.486251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-15f23fe8-6238-49e4-8259-7926f28de9ff None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:42.487470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-15f23fe8-6238-49e4-8259-7926f28de9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:42.495751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 931/1853] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:42.500603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9d2f4-0e9c-4b9b-964f-649efa5ff0ea None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:42.506866 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:42.512701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9d2f4-0e9c-4b9b-964f-649efa5ff0ea None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:42.519905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc14be-80ab-471a-b332-cf4d54dccd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:42.520637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc14be-80ab-471a-b332-cf4d54dccd7b admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:42.521098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc14be-80ab-471a-b332-cf4d54dccd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:42.521534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc14be-80ab-471a-b332-cf4d54dccd7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:42.522402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc14be-80ab-471a-b332-cf4d54dccd7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:42.522708 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc14be-80ab-471a-b332-cf4d54dccd7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:42.524299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc14be-80ab-471a-b332-cf4d54dccd7b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:42.536819 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 923/1854] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:42.540568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f23fe8-6238-49e4-8259-7926f28de9ff None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:42.549783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f23fe8-6238-49e4-8259-7926f28de9ff None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:42.554472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bee66-21c2-44f7-9c35-b24c899ad81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:42.555063 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bee66-21c2-44f7-9c35-b24c899ad81c admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:42.555334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bee66-21c2-44f7-9c35-b24c899ad81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:42.555599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bee66-21c2-44f7-9c35-b24c899ad81c admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:42.556363 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bee66-21c2-44f7-9c35-b24c899ad81c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:42.558451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bee66-21c2-44f7-9c35-b24c899ad81c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:42.587138 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 932/1855] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:42 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:42.591375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc14be-80ab-471a-b332-cf4d54dccd7b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:42.602102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc14be-80ab-471a-b332-cf4d54dccd7b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:42.664030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9062e-fcec-4202-bbdd-80b28dcabc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:42.664645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9062e-fcec-4202-bbdd-80b28dcabc8b admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:42.664923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9062e-fcec-4202-bbdd-80b28dcabc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:42.665194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9062e-fcec-4202-bbdd-80b28dcabc8b admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:42.665824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9062e-fcec-4202-bbdd-80b28dcabc8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:42.667908 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9062e-fcec-4202-bbdd-80b28dcabc8b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:42.691726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 924/1856] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:42 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:42.696932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bee66-21c2-44f7-9c35-b24c899ad81c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:42.707464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bee66-21c2-44f7-9c35-b24c899ad81c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:42.712176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbc35c-8e2b-4590-bc96-fc468009c890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:42.712731 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbc35c-8e2b-4590-bc96-fc468009c890 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:42.713041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbc35c-8e2b-4590-bc96-fc468009c890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:42.713308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbc35c-8e2b-4590-bc96-fc468009c890 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:42.713944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbc35c-8e2b-4590-bc96-fc468009c890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:42.715688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbc35c-8e2b-4590-bc96-fc468009c890 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:42.726090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 933/1857] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:42.730225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9062e-fcec-4202-bbdd-80b28dcabc8b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:42.741203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9062e-fcec-4202-bbdd-80b28dcabc8b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:42.745995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21423a6f-6ef2-4690-af6e-9fac28cd43cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:42.746650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21423a6f-6ef2-4690-af6e-9fac28cd43cd admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:42.746930 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21423a6f-6ef2-4690-af6e-9fac28cd43cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:42.747196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21423a6f-6ef2-4690-af6e-9fac28cd43cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:42.747836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21423a6f-6ef2-4690-af6e-9fac28cd43cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:42.749417 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21423a6f-6ef2-4690-af6e-9fac28cd43cd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:42.762007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 925/1858] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:42.765534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbc35c-8e2b-4590-bc96-fc468009c890 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:42.775734 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbc35c-8e2b-4590-bc96-fc468009c890 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:42.780339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7327a3-0c77-44bf-b81f-8e157c2ca991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:42.780987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7327a3-0c77-44bf-b81f-8e157c2ca991 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:42.781292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7327a3-0c77-44bf-b81f-8e157c2ca991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:42.781566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7327a3-0c77-44bf-b81f-8e157c2ca991 admin admin] PATH_INFO: `/v3/projects/a578ee882e60456392c88a508d1a6db3/users/16228db2ddd147969482871b600ac797/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:42.796268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7327a3-0c77-44bf-b81f-8e157c2ca991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a578ee882e60456392c88a508d1a6db3, user_id=16228db2ddd147969482871b600ac797, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:42.799669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7327a3-0c77-44bf-b81f-8e157c2ca991 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:42.812527 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 934/1859] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:42 2022] PUT /identity/v3/projects/a578ee882e60456392c88a508d1a6db3/users/16228db2ddd147969482871b600ac797/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:42.818046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88980ef0-a95c-47c3-92ce-bbbcc88a2d8a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:42.818364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88980ef0-a95c-47c3-92ce-bbbcc88a2d8a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:42.818638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88980ef0-a95c-47c3-92ce-bbbcc88a2d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:42.838390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-88980ef0-a95c-47c3-92ce-bbbcc88a2d8a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:42.839598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-88980ef0-a95c-47c3-92ce-bbbcc88a2d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:42.899571 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 926/1860] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:42.909374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7327a3-0c77-44bf-b81f-8e157c2ca991 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:42.911135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0a7327a3-0c77-44bf-b81f-8e157c2ca991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:42.971434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7327a3-0c77-44bf-b81f-8e157c2ca991 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:42.975104 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52215c-eac1-49cc-a239-48806e666b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:42.975712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52215c-eac1-49cc-a239-48806e666b00 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:42.975989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52215c-eac1-49cc-a239-48806e666b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:42.976258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52215c-eac1-49cc-a239-48806e666b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:42.985069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52215c-eac1-49cc-a239-48806e666b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:42.987009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52215c-eac1-49cc-a239-48806e666b00 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:42.996401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 935/1861] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:47.476063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b85786-5528-42d7-8c7e-ede3b634032d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:47.476576 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b85786-5528-42d7-8c7e-ede3b634032d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:47.476875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b85786-5528-42d7-8c7e-ede3b634032d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:47.510626 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-96b85786-5528-42d7-8c7e-ede3b634032d None None] MFA Rules not processed for user `16228db2ddd147969482871b600ac797`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:47.511849 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-96b85786-5528-42d7-8c7e-ede3b634032d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:47.597033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:47.604944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 927/1862] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:47.618634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52215c-eac1-49cc-a239-48806e666b00 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:47.620054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ab52215c-eac1-49cc-a239-48806e666b00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:47.682988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:47.717751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52215c-eac1-49cc-a239-48806e666b00 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:47.723321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e539f6e-c59c-41ba-8ae3-5ee6436c20df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:47.723931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e539f6e-c59c-41ba-8ae3-5ee6436c20df service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:47.724211 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e539f6e-c59c-41ba-8ae3-5ee6436c20df service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:47.724479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e539f6e-c59c-41ba-8ae3-5ee6436c20df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:47.732287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e539f6e-c59c-41ba-8ae3-5ee6436c20df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:47.733857 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e539f6e-c59c-41ba-8ae3-5ee6436c20df service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:47.742955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 936/1863] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:49.289190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031483cb-7d9d-483f-8a6f-5b2ba4aae887 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:49.289731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031483cb-7d9d-483f-8a6f-5b2ba4aae887 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:49.289846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031483cb-7d9d-483f-8a6f-5b2ba4aae887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:49.308643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-031483cb-7d9d-483f-8a6f-5b2ba4aae887 None None] MFA Rules not processed for user `16228db2ddd147969482871b600ac797`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:49.310286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-031483cb-7d9d-483f-8a6f-5b2ba4aae887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:49.318337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 928/1864] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:54.087244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e539f6e-c59c-41ba-8ae3-5ee6436c20df service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:54.089166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4e539f6e-c59c-41ba-8ae3-5ee6436c20df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:54.103034 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e539f6e-c59c-41ba-8ae3-5ee6436c20df service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:54.108256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:54.108883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:54.109366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:54.109639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] PATH_INFO: `/v3/users/528e019cd07148f2b85566c90b1f1be9` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:54.111669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=528e019cd07148f2b85566c90b1f1be9)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:54.113869 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:54.189428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '528e019cd07148f2b85566c90b1f1be9'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:54.190330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '528e019cd07148f2b85566c90b1f1be9'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:54.226502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '528e019cd07148f2b85566c90b1f1be9'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:54.228821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '528e019cd07148f2b85566c90b1f1be9'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:54.247686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '528e019cd07148f2b85566c90b1f1be9'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:54.273708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 937/1865] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:54 2022] DELETE /identity/v3/users/528e019cd07148f2b85566c90b1f1be9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:54.278122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49efcb0-bdb6-4b87-84d5-16a0f460fad8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:54.279703 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49efcb0-bdb6-4b87-84d5-16a0f460fad8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:54.280169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49efcb0-bdb6-4b87-84d5-16a0f460fad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:54.313140 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d49efcb0-bdb6-4b87-84d5-16a0f460fad8 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:54.315401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d49efcb0-bdb6-4b87-84d5-16a0f460fad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:54.404077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 929/1866] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:54.437833 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:54.439637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:54.529361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320d583-5b0e-4f5d-af4a-4f2ccef82dd1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:54.534025 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9ff22-6dff-47a6-9646-f1eae61cfb38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:54.534900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9ff22-6dff-47a6-9646-f1eae61cfb38 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:54.535180 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9ff22-6dff-47a6-9646-f1eae61cfb38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:54.535455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9ff22-6dff-47a6-9646-f1eae61cfb38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:54.544791 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9ff22-6dff-47a6-9646-f1eae61cfb38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:54.546228 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9ff22-6dff-47a6-9646-f1eae61cfb38 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:54.557699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 938/1867] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:54.848251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49efcb0-bdb6-4b87-84d5-16a0f460fad8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:54.849815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d49efcb0-bdb6-4b87-84d5-16a0f460fad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:54.861543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49efcb0-bdb6-4b87-84d5-16a0f460fad8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:54.873604 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:54.874607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:54.874919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:54.875191 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] PATH_INFO: `/v3/projects/3a23d7348eb742348f46e7a802559cc7` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:54.882221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a23d7348eb742348f46e7a802559cc7)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:54.884964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:54.952840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a23d7348eb742348f46e7a802559cc7', 'request_id': 'req-e33c373e-635e-4568-92ed-c7897c64a090'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:54.963421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a23d7348eb742348f46e7a802559cc7', 'request_id': 'req-e33c373e-635e-4568-92ed-c7897c64a090'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:54.970822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a23d7348eb742348f46e7a802559cc7', 'request_id': 'req-e33c373e-635e-4568-92ed-c7897c64a090'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:54.984782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] The token cache is being invalidate because project 3a23d7348eb742348f46e7a802559cc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:54.985070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:54.987077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e33c373e-635e-4568-92ed-c7897c64a090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:54.988601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 930/1868] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:54 2022] DELETE /identity/v3/projects/3a23d7348eb742348f46e7a802559cc7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:04.557996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=72,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:04.981435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:09.644642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9ff22-6dff-47a6-9646-f1eae61cfb38 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:09.646076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-68b9ff22-6dff-47a6-9646-f1eae61cfb38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:09.659629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9ff22-6dff-47a6-9646-f1eae61cfb38 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:09.718515 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:09.719061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:09.719336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:09.719600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] PATH_INFO: `/v3/users/083c3a810470453f831183843ba03707` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:09.720698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] RBAC: Authorizing `identity:delete_user(user_id=083c3a810470453f831183843ba03707)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:09.723678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:09.769186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083c3a810470453f831183843ba03707'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:09.769834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083c3a810470453f831183843ba03707'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:09.781997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083c3a810470453f831183843ba03707'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:09.782686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083c3a810470453f831183843ba03707'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:09.793790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083c3a810470453f831183843ba03707'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:09.809179 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 939/1869] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:44:09 2022] DELETE /identity/v3/users/083c3a810470453f831183843ba03707 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:09.813682 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8ff5c-9727-49df-b583-16c51d331a86 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:09.814155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8ff5c-9727-49df-b583-16c51d331a86 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:09.814283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8ff5c-9727-49df-b583-16c51d331a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:09.834594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-84e8ff5c-9727-49df-b583-16c51d331a86 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:09.835791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-84e8ff5c-9727-49df-b583-16c51d331a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:09.897552 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 931/1870] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:09.907207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:09.908875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:09.968767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee785e-8608-4d59-b2bf-8f4dc1f3919f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:09.972963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab9b12-3aae-4b3a-bc21-123e0c417617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:09.973565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab9b12-3aae-4b3a-bc21-123e0c417617 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:09.973838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab9b12-3aae-4b3a-bc21-123e0c417617 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:09.974098 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab9b12-3aae-4b3a-bc21-123e0c417617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:09.983161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab9b12-3aae-4b3a-bc21-123e0c417617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:09.984603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab9b12-3aae-4b3a-bc21-123e0c417617 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:09.994278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 940/1871] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:10.324715 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8ff5c-9727-49df-b583-16c51d331a86 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:10.326237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-84e8ff5c-9727-49df-b583-16c51d331a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:10.339393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8ff5c-9727-49df-b583-16c51d331a86 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:10.344793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:10.345551 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.345895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.346275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] PATH_INFO: `/v3/projects/7930734bac2c4182bc42e6ce3112c646` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.347845 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7930734bac2c4182bc42e6ce3112c646)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:10.349814 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:10.408878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7930734bac2c4182bc42e6ce3112c646', 'request_id': 'req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:10.417633 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7930734bac2c4182bc42e6ce3112c646', 'request_id': 'req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:10.425129 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7930734bac2c4182bc42e6ce3112c646', 'request_id': 'req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:10.435730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] The token cache is being invalidate because project 7930734bac2c4182bc42e6ce3112c646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:44:10.436017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:10.436663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7b7c2f4f-5a32-415b-be4f-f4f9cd28b207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:10.438156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 932/1872] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:44:10 2022] DELETE /identity/v3/projects/7930734bac2c4182bc42e6ce3112c646 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:10.443761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b727d-9cc1-495d-ad58-ff1535b15c2b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.444051 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b727d-9cc1-495d-ad58-ff1535b15c2b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.444404 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b727d-9cc1-495d-ad58-ff1535b15c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.464902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-276b727d-9cc1-495d-ad58-ff1535b15c2b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:10.466143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-276b727d-9cc1-495d-ad58-ff1535b15c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:10.482963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:10.529829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 941/1873] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:10.538143 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40053f9-7444-4768-9ac4-e247aa9b21fa None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.538474 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40053f9-7444-4768-9ac4-e247aa9b21fa None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.538752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40053f9-7444-4768-9ac4-e247aa9b21fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.560736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c40053f9-7444-4768-9ac4-e247aa9b21fa None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:10.562135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c40053f9-7444-4768-9ac4-e247aa9b21fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:10.574776 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 933/1874] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:10.575805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b727d-9cc1-495d-ad58-ff1535b15c2b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:10.587370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b727d-9cc1-495d-ad58-ff1535b15c2b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:10.592575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb846394-39a5-4c5c-9208-ef084c0d112f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:10.593265 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb846394-39a5-4c5c-9208-ef084c0d112f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.593621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb846394-39a5-4c5c-9208-ef084c0d112f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.593992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb846394-39a5-4c5c-9208-ef084c0d112f admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.594686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb846394-39a5-4c5c-9208-ef084c0d112f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:44:10.595023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb846394-39a5-4c5c-9208-ef084c0d112f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:10.596643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb846394-39a5-4c5c-9208-ef084c0d112f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:10.608775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 942/1875] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:44:10.613940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40053f9-7444-4768-9ac4-e247aa9b21fa None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:10.623469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40053f9-7444-4768-9ac4-e247aa9b21fa None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:10.629104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c7b66-5bc8-4ecc-ac5b-f8b0ad2f5abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:10.629705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c7b66-5bc8-4ecc-ac5b-f8b0ad2f5abf admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.629994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c7b66-5bc8-4ecc-ac5b-f8b0ad2f5abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.630266 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c7b66-5bc8-4ecc-ac5b-f8b0ad2f5abf admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.630901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c7b66-5bc8-4ecc-ac5b-f8b0ad2f5abf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:10.632971 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c7b66-5bc8-4ecc-ac5b-f8b0ad2f5abf admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:10.669642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 934/1876] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:44:10 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:10.671170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb846394-39a5-4c5c-9208-ef084c0d112f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:10.682352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb846394-39a5-4c5c-9208-ef084c0d112f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:10.744038 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fffa3-38b2-4ab6-b55d-2739339bd63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:10.744655 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fffa3-38b2-4ab6-b55d-2739339bd63e admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.744937 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fffa3-38b2-4ab6-b55d-2739339bd63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.745208 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fffa3-38b2-4ab6-b55d-2739339bd63e admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.745837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fffa3-38b2-4ab6-b55d-2739339bd63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:10.748015 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fffa3-38b2-4ab6-b55d-2739339bd63e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:10.774572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 943/1877] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:44:10 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:10.778611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c7b66-5bc8-4ecc-ac5b-f8b0ad2f5abf admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:10.801436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c7b66-5bc8-4ecc-ac5b-f8b0ad2f5abf admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:10.806994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133eed29-0170-450b-bdb7-fe87269b174f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:10.807709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133eed29-0170-450b-bdb7-fe87269b174f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.808060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133eed29-0170-450b-bdb7-fe87269b174f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.808403 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133eed29-0170-450b-bdb7-fe87269b174f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.809267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133eed29-0170-450b-bdb7-fe87269b174f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:10.811063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133eed29-0170-450b-bdb7-fe87269b174f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:10.825182 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 935/1878] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:10.826483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fffa3-38b2-4ab6-b55d-2739339bd63e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:10.836686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fffa3-38b2-4ab6-b55d-2739339bd63e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:10.841527 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e22947-530e-47f3-acce-8067caa5a7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:10.842223 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22947-530e-47f3-acce-8067caa5a7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.842567 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22947-530e-47f3-acce-8067caa5a7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.842906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22947-530e-47f3-acce-8067caa5a7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.843641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e22947-530e-47f3-acce-8067caa5a7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:10.845502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e22947-530e-47f3-acce-8067caa5a7f0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:10.857853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 944/1879] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:44:10.862105 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133eed29-0170-450b-bdb7-fe87269b174f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:10.873706 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133eed29-0170-450b-bdb7-fe87269b174f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:10.878722 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea7477-0f0e-41ba-bd94-2a5fafdba248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:10.879504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea7477-0f0e-41ba-bd94-2a5fafdba248 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.879710 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea7477-0f0e-41ba-bd94-2a5fafdba248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.879934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea7477-0f0e-41ba-bd94-2a5fafdba248 admin admin] PATH_INFO: `/v3/projects/0bc5e5a0d29e43589d95be16c1fb464e/users/8bdfee122af8429c9b8766949b781aee/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.899284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea7477-0f0e-41ba-bd94-2a5fafdba248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bc5e5a0d29e43589d95be16c1fb464e, user_id=8bdfee122af8429c9b8766949b781aee, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:10.902995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea7477-0f0e-41ba-bd94-2a5fafdba248 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:10.921509 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 936/1880] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:10 2022] PUT /identity/v3/projects/0bc5e5a0d29e43589d95be16c1fb464e/users/8bdfee122af8429c9b8766949b781aee/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:10.927601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82a509-28c2-49f9-a7f2-176e28b1b3f5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.927885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82a509-28c2-49f9-a7f2-176e28b1b3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.928161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82a509-28c2-49f9-a7f2-176e28b1b3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.947843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9d82a509-28c2-49f9-a7f2-176e28b1b3f5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:10.949046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9d82a509-28c2-49f9-a7f2-176e28b1b3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:11.013544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 945/1881] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:44:11.024053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea7477-0f0e-41ba-bd94-2a5fafdba248 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:11.026585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-33ea7477-0f0e-41ba-bd94-2a5fafdba248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:11.049329 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:11.087817 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea7477-0f0e-41ba-bd94-2a5fafdba248 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:11.091467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c2033-c8ff-4908-82ec-20b2541a8050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:11.092077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c2033-c8ff-4908-82ec-20b2541a8050 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:11.092355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c2033-c8ff-4908-82ec-20b2541a8050 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:11.092626 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c2033-c8ff-4908-82ec-20b2541a8050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:11.101886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c2033-c8ff-4908-82ec-20b2541a8050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:11.103548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c2033-c8ff-4908-82ec-20b2541a8050 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:11.115629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 937/1882] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:44:12.016084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9897c09-ad77-4c74-a9cb-6ff67650cf04 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:12.016657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9897c09-ad77-4c74-a9cb-6ff67650cf04 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:12.016657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9897c09-ad77-4c74-a9cb-6ff67650cf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:12.061117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f9897c09-ad77-4c74-a9cb-6ff67650cf04 None None] MFA Rules not processed for user `8bdfee122af8429c9b8766949b781aee`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:12.062475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f9897c09-ad77-4c74-a9cb-6ff67650cf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:12.083032 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:12.142986 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 946/1883] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:12.155744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c2033-c8ff-4908-82ec-20b2541a8050 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:12.158234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e65c2033-c8ff-4908-82ec-20b2541a8050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:12.258307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c2033-c8ff-4908-82ec-20b2541a8050 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:12.262451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfbbf0-be8f-4031-8407-0aa26f72bbcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:12.263075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfbbf0-be8f-4031-8407-0aa26f72bbcc service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:12.263352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfbbf0-be8f-4031-8407-0aa26f72bbcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:12.263802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfbbf0-be8f-4031-8407-0aa26f72bbcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:12.273627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfbbf0-be8f-4031-8407-0aa26f72bbcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:12.275123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfbbf0-be8f-4031-8407-0aa26f72bbcc service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:12.285621 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 938/1884] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:13.707252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64b979-7ef4-4a26-93cc-bd5d0f0da279 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:13.707803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64b979-7ef4-4a26-93cc-bd5d0f0da279 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:13.707803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64b979-7ef4-4a26-93cc-bd5d0f0da279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:13.728560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-eb64b979-7ef4-4a26-93cc-bd5d0f0da279 None None] MFA Rules not processed for user `8bdfee122af8429c9b8766949b781aee`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:13.729766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-eb64b979-7ef4-4a26-93cc-bd5d0f0da279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:13.743654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 947/1885] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:18.350467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfbbf0-be8f-4031-8407-0aa26f72bbcc service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:18.351869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-fccfbbf0-be8f-4031-8407-0aa26f72bbcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:18.365260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfbbf0-be8f-4031-8407-0aa26f72bbcc service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:18.369671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:18.370309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:18.370589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:18.370852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] PATH_INFO: `/v3/users/bb3858c54a4842239150d8645bdafbf2` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:18.371929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb3858c54a4842239150d8645bdafbf2)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:18.373814 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:18.421735 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb3858c54a4842239150d8645bdafbf2'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:18.422414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb3858c54a4842239150d8645bdafbf2'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:18.423038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb3858c54a4842239150d8645bdafbf2'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:18.429960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb3858c54a4842239150d8645bdafbf2'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:18.442361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb3858c54a4842239150d8645bdafbf2'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:18.454672 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 939/1886] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:44:18 2022] DELETE /identity/v3/users/bb3858c54a4842239150d8645bdafbf2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:18.459809 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953772-3232-4fb3-bbbe-052ac5d95e20 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:18.460270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953772-3232-4fb3-bbbe-052ac5d95e20 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:18.460361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953772-3232-4fb3-bbbe-052ac5d95e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:18.479506 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f7953772-3232-4fb3-bbbe-052ac5d95e20 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:18.480709 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f7953772-3232-4fb3-bbbe-052ac5d95e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:18.539071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 948/1887] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:18.548773 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:18.550722 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:18.593036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:18.609808 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d71c8-9157-448e-90a9-76aae7cc6b4e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:18.614480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c14151-26ac-46c1-87ea-28f3c8dadb16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:18.615178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c14151-26ac-46c1-87ea-28f3c8dadb16 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:18.615459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c14151-26ac-46c1-87ea-28f3c8dadb16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:18.615730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c14151-26ac-46c1-87ea-28f3c8dadb16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:18.624500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c14151-26ac-46c1-87ea-28f3c8dadb16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:18.625912 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c14151-26ac-46c1-87ea-28f3c8dadb16 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:18.636403 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 940/1888] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:18.905263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7953772-3232-4fb3-bbbe-052ac5d95e20 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:18.906836 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f7953772-3232-4fb3-bbbe-052ac5d95e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:18.920681 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7953772-3232-4fb3-bbbe-052ac5d95e20 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:18.927087 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:18.927745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:18.928021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:18.928291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] PATH_INFO: `/v3/projects/c0c63e842bab45b3aaaf36257175c04c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:18.929409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0c63e842bab45b3aaaf36257175c04c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:18.931352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:18.978000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0c63e842bab45b3aaaf36257175c04c', 'request_id': 'req-376d54b5-4f2d-41c0-aaba-dd3e57109593'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:18.983697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0c63e842bab45b3aaaf36257175c04c', 'request_id': 'req-376d54b5-4f2d-41c0-aaba-dd3e57109593'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:18.991572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0c63e842bab45b3aaaf36257175c04c', 'request_id': 'req-376d54b5-4f2d-41c0-aaba-dd3e57109593'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:19.002658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] The token cache is being invalidate because project c0c63e842bab45b3aaaf36257175c04c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:44:19.003124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:19.003711 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-376d54b5-4f2d-41c0-aaba-dd3e57109593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:19.005243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 949/1889] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:44:18 2022] DELETE /identity/v3/projects/c0c63e842bab45b3aaaf36257175c04c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:19.010337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b8f0c-49b8-4db7-97f6-6cb764596850 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:19.010631 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b8f0c-49b8-4db7-97f6-6cb764596850 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:19.010910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b8f0c-49b8-4db7-97f6-6cb764596850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:19.030066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-892b8f0c-49b8-4db7-97f6-6cb764596850 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:19.031281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-892b8f0c-49b8-4db7-97f6-6cb764596850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:19.110848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 941/1890] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:19.128934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904a307-71f2-4157-817c-b6ad8eaa7675 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:19.128934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904a307-71f2-4157-817c-b6ad8eaa7675 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:19.128934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904a307-71f2-4157-817c-b6ad8eaa7675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:19.146227 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f904a307-71f2-4157-817c-b6ad8eaa7675 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:19.147633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f904a307-71f2-4157-817c-b6ad8eaa7675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:19.163768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 950/1891] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:19.165009 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b8f0c-49b8-4db7-97f6-6cb764596850 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:19.177920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b8f0c-49b8-4db7-97f6-6cb764596850 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:19.184841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d011ad-c9f7-4c35-b3e0-83db31f0817c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:19.185599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d011ad-c9f7-4c35-b3e0-83db31f0817c admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:19.186082 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d011ad-c9f7-4c35-b3e0-83db31f0817c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:19.186505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d011ad-c9f7-4c35-b3e0-83db31f0817c admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:19.187259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d011ad-c9f7-4c35-b3e0-83db31f0817c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:44:19.188411 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d011ad-c9f7-4c35-b3e0-83db31f0817c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:19.190401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d011ad-c9f7-4c35-b3e0-83db31f0817c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:19.218984 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 942/1892] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:44:19.224181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904a307-71f2-4157-817c-b6ad8eaa7675 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:19.233724 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904a307-71f2-4157-817c-b6ad8eaa7675 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:19.239760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d443ed-13a1-4b14-b71e-d5e5a30321f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:19.240537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d443ed-13a1-4b14-b71e-d5e5a30321f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:19.240912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d443ed-13a1-4b14-b71e-d5e5a30321f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:19.241279 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d443ed-13a1-4b14-b71e-d5e5a30321f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:19.242023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d443ed-13a1-4b14-b71e-d5e5a30321f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:19.244041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d443ed-13a1-4b14-b71e-d5e5a30321f3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:19.283232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 951/1893] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:44:19 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:44:19.284647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d011ad-c9f7-4c35-b3e0-83db31f0817c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:19.294878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d011ad-c9f7-4c35-b3e0-83db31f0817c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:19.359537 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913425fe-551c-4d42-b5bd-2d0d70e14de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:19.360128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913425fe-551c-4d42-b5bd-2d0d70e14de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:19.360128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913425fe-551c-4d42-b5bd-2d0d70e14de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:19.360315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913425fe-551c-4d42-b5bd-2d0d70e14de8 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:19.361002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913425fe-551c-4d42-b5bd-2d0d70e14de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:19.363775 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913425fe-551c-4d42-b5bd-2d0d70e14de8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:19.389960 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 943/1894] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:44:19 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:44:19.393976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d443ed-13a1-4b14-b71e-d5e5a30321f3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:19.404262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d443ed-13a1-4b14-b71e-d5e5a30321f3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:19.409442 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1ce24-952f-478f-b763-2cd2d9e8a5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:19.410283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1ce24-952f-478f-b763-2cd2d9e8a5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:19.410566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1ce24-952f-478f-b763-2cd2d9e8a5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:19.410838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1ce24-952f-478f-b763-2cd2d9e8a5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:19.411499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1ce24-952f-478f-b763-2cd2d9e8a5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:19.413069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1ce24-952f-478f-b763-2cd2d9e8a5f3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:19.414687 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:19.426907 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 952/1895] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:19.430136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913425fe-551c-4d42-b5bd-2d0d70e14de8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:19.442103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913425fe-551c-4d42-b5bd-2d0d70e14de8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:19.447267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d4405-e14f-42b3-b135-5ccb788324fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:19.447876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d4405-e14f-42b3-b135-5ccb788324fa admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:19.448153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d4405-e14f-42b3-b135-5ccb788324fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:19.448475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d4405-e14f-42b3-b135-5ccb788324fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:19.449065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d4405-e14f-42b3-b135-5ccb788324fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:19.450724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d4405-e14f-42b3-b135-5ccb788324fa admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:19.462972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 944/1896] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:19.467231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1ce24-952f-478f-b763-2cd2d9e8a5f3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:19.477622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1ce24-952f-478f-b763-2cd2d9e8a5f3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:19.482702 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c46e12-8664-4e6a-8bc5-7517382b770a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:19.483358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c46e12-8664-4e6a-8bc5-7517382b770a admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:19.483632 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c46e12-8664-4e6a-8bc5-7517382b770a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:19.483896 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c46e12-8664-4e6a-8bc5-7517382b770a admin admin] PATH_INFO: `/v3/projects/5243c296c8cb4467bfd269e4622e1adb/users/6e926bb8de284795bb6ebbda2dccc266/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:19.500880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c46e12-8664-4e6a-8bc5-7517382b770a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5243c296c8cb4467bfd269e4622e1adb, user_id=6e926bb8de284795bb6ebbda2dccc266, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:19.504353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c46e12-8664-4e6a-8bc5-7517382b770a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:19.519128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 953/1897] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:19 2022] PUT /identity/v3/projects/5243c296c8cb4467bfd269e4622e1adb/users/6e926bb8de284795bb6ebbda2dccc266/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:19.524061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe881a-0abc-4983-bfb9-2b78b525acc6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:19.524434 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe881a-0abc-4983-bfb9-2b78b525acc6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:19.524727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe881a-0abc-4983-bfb9-2b78b525acc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:19.545825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3cfe881a-0abc-4983-bfb9-2b78b525acc6 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:19.547056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3cfe881a-0abc-4983-bfb9-2b78b525acc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:19.590258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:19.609039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 945/1898] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:44:19.619223 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c46e12-8664-4e6a-8bc5-7517382b770a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:19.620915 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a4c46e12-8664-4e6a-8bc5-7517382b770a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:19.676896 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c46e12-8664-4e6a-8bc5-7517382b770a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:19.680801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767053d9-d637-4af5-b06f-8cd2ead63398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:19.681231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767053d9-d637-4af5-b06f-8cd2ead63398 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:19.681540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767053d9-d637-4af5-b06f-8cd2ead63398 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:19.681851 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767053d9-d637-4af5-b06f-8cd2ead63398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:19.692961 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767053d9-d637-4af5-b06f-8cd2ead63398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:19.694456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767053d9-d637-4af5-b06f-8cd2ead63398 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:19.703898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 954/1899] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:24.099426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab98fee-9e36-4269-9785-d78a2bb77d86 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:24.099951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab98fee-9e36-4269-9785-d78a2bb77d86 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:24.099951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab98fee-9e36-4269-9785-d78a2bb77d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:24.147453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2ab98fee-9e36-4269-9785-d78a2bb77d86 None None] MFA Rules not processed for user `6e926bb8de284795bb6ebbda2dccc266`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:24.149477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2ab98fee-9e36-4269-9785-d78a2bb77d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:24.247379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 946/1900] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:24.252803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767053d9-d637-4af5-b06f-8cd2ead63398 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:24.263344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767053d9-d637-4af5-b06f-8cd2ead63398 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:24.270451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ddc010-21f7-4bb6-a2f1-0c1026abeb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:24.271081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddc010-21f7-4bb6-a2f1-0c1026abeb41 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:24.271622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddc010-21f7-4bb6-a2f1-0c1026abeb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:24.271897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddc010-21f7-4bb6-a2f1-0c1026abeb41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:24.272542 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ddc010-21f7-4bb6-a2f1-0c1026abeb41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:24.274484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ddc010-21f7-4bb6-a2f1-0c1026abeb41 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:24.308097 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 955/1901] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:44:24 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:24.312065 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab98fee-9e36-4269-9785-d78a2bb77d86 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:24.337521 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab98fee-9e36-4269-9785-d78a2bb77d86 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:24.400578 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6e5e8-0e69-4499-b19a-a4d1c28debd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:24.401203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e5e8-0e69-4499-b19a-a4d1c28debd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:24.401484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e5e8-0e69-4499-b19a-a4d1c28debd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:24.401757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6e5e8-0e69-4499-b19a-a4d1c28debd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:24.402447 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6e5e8-0e69-4499-b19a-a4d1c28debd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:24.404377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6e5e8-0e69-4499-b19a-a4d1c28debd7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:24.430130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 947/1902] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:44:24 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:24.431434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ddc010-21f7-4bb6-a2f1-0c1026abeb41 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:24.442609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ddc010-21f7-4bb6-a2f1-0c1026abeb41 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:24.448139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51f6a0-62d2-4e47-8d59-f175c1a0826d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:24.448759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51f6a0-62d2-4e47-8d59-f175c1a0826d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:24.449035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51f6a0-62d2-4e47-8d59-f175c1a0826d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:24.449482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51f6a0-62d2-4e47-8d59-f175c1a0826d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:24.450125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51f6a0-62d2-4e47-8d59-f175c1a0826d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:24.451946 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51f6a0-62d2-4e47-8d59-f175c1a0826d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:24.470084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 956/1903] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:24.471216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6e5e8-0e69-4499-b19a-a4d1c28debd7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:24.483843 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6e5e8-0e69-4499-b19a-a4d1c28debd7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:24.489825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddaf1b-ca6b-4b55-afbd-5537831805fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:24.490622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddaf1b-ca6b-4b55-afbd-5537831805fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:24.490976 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddaf1b-ca6b-4b55-afbd-5537831805fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:24.491406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddaf1b-ca6b-4b55-afbd-5537831805fa admin admin] PATH_INFO: `/v3/projects/76432deb97844d819f955521555aa08c/users/264583661f414bdfafc3f0a14faccf44/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:24.525547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddaf1b-ca6b-4b55-afbd-5537831805fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=76432deb97844d819f955521555aa08c, user_id=264583661f414bdfafc3f0a14faccf44, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:24.528508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddaf1b-ca6b-4b55-afbd-5537831805fa admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:24.555327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 948/1904] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:24 2022] PUT /identity/v3/projects/76432deb97844d819f955521555aa08c/users/264583661f414bdfafc3f0a14faccf44/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:44:24.559230 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51f6a0-62d2-4e47-8d59-f175c1a0826d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:24.570047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51f6a0-62d2-4e47-8d59-f175c1a0826d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:24.654391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfaa73-64ce-405a-9206-15207177b22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:24.655262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfaa73-64ce-405a-9206-15207177b22d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:24.664097 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfaa73-64ce-405a-9206-15207177b22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:24.664445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfaa73-64ce-405a-9206-15207177b22d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:24.665174 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedfaa73-64ce-405a-9206-15207177b22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:24.666828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedfaa73-64ce-405a-9206-15207177b22d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:24.678384 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 957/1905] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:24.682633 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddaf1b-ca6b-4b55-afbd-5537831805fa admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:24.693942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddaf1b-ca6b-4b55-afbd-5537831805fa admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:24.699124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc2cf6-1f63-4afd-85f0-8de6eb6814ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:24.699795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc2cf6-1f63-4afd-85f0-8de6eb6814ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:24.700080 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc2cf6-1f63-4afd-85f0-8de6eb6814ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:24.700348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc2cf6-1f63-4afd-85f0-8de6eb6814ef admin admin] PATH_INFO: `/v3/projects/76432deb97844d819f955521555aa08c/users/264583661f414bdfafc3f0a14faccf44/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:24.702307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc2cf6-1f63-4afd-85f0-8de6eb6814ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=76432deb97844d819f955521555aa08c, user_id=264583661f414bdfafc3f0a14faccf44, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:24.704936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc2cf6-1f63-4afd-85f0-8de6eb6814ef admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:24.716485 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 949/1906] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:24 2022] PUT /identity/v3/projects/76432deb97844d819f955521555aa08c/users/264583661f414bdfafc3f0a14faccf44/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:29.726280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6522d56-003c-446d-84da-66dd87a5ee7c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:29.726702 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6522d56-003c-446d-84da-66dd87a5ee7c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:29.726796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6522d56-003c-446d-84da-66dd87a5ee7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:29.756127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f6522d56-003c-446d-84da-66dd87a5ee7c None None] MFA Rules not processed for user `264583661f414bdfafc3f0a14faccf44`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:29.757307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f6522d56-003c-446d-84da-66dd87a5ee7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:29.785242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:29.831523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 958/1907] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:29.844294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc2cf6-1f63-4afd-85f0-8de6eb6814ef admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:29.846647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b8cc2cf6-1f63-4afd-85f0-8de6eb6814ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:29.879339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:29.909798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc2cf6-1f63-4afd-85f0-8de6eb6814ef admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:29.913689 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e967361-bc1e-4c1b-b6a4-57d6316e9487 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07f8443d5594f47bc81e9f9dc9ce43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:29.914334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e967361-bc1e-4c1b-b6a4-57d6316e9487 service glance] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:29.914612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e967361-bc1e-4c1b-b6a4-57d6316e9487 service glance] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:29.914878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e967361-bc1e-4c1b-b6a4-57d6316e9487 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:29.979257 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e967361-bc1e-4c1b-b6a4-57d6316e9487 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:29.981004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e967361-bc1e-4c1b-b6a4-57d6316e9487 service glance] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:29.991802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 950/1908] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:30.167968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011d658-a265-44a5-960e-e5f8fd29d2fc None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:30.168310 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011d658-a265-44a5-960e-e5f8fd29d2fc None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:30.168593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011d658-a265-44a5-960e-e5f8fd29d2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:30.193332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-4011d658-a265-44a5-960e-e5f8fd29d2fc None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:30.195059 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4011d658-a265-44a5-960e-e5f8fd29d2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:30.285043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 959/1909] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:44:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:30.340670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e967361-bc1e-4c1b-b6a4-57d6316e9487 service glance] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:30.342069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9e967361-bc1e-4c1b-b6a4-57d6316e9487 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:30.426042 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e967361-bc1e-4c1b-b6a4-57d6316e9487 service glance] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:30.440299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ef6c9-0f2e-4de7-aef2-f375ca5eee08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:30.440299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ef6c9-0f2e-4de7-aef2-f375ca5eee08 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:30.440581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ef6c9-0f2e-4de7-aef2-f375ca5eee08 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:30.440920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ef6c9-0f2e-4de7-aef2-f375ca5eee08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:30.481463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ef6c9-0f2e-4de7-aef2-f375ca5eee08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:30.483673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ef6c9-0f2e-4de7-aef2-f375ca5eee08 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:30.495875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 951/1910] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:44:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:44:33.756847 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7738e1-8ccf-4915-9a40-10abe9f2be27 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:33.757406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7738e1-8ccf-4915-9a40-10abe9f2be27 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:33.757496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7738e1-8ccf-4915-9a40-10abe9f2be27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:33.790631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-6a7738e1-8ccf-4915-9a40-10abe9f2be27 None None] MFA Rules not processed for user `be793f4685f94beba74adedbc02ee50a`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:33.792698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6a7738e1-8ccf-4915-9a40-10abe9f2be27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:33.869674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 960/1911] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:40.491886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=67 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:41.338039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19efde-efa3-4b13-84ff-a998195b5f7e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:41.338396 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19efde-efa3-4b13-84ff-a998195b5f7e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:41.338678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19efde-efa3-4b13-84ff-a998195b5f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:41.364262 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-de19efde-efa3-4b13-84ff-a998195b5f7e None None] MFA Rules not processed for user `6e926bb8de284795bb6ebbda2dccc266`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:41.365620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-de19efde-efa3-4b13-84ff-a998195b5f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:41.373999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 952/1912] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:42.445003 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6de7a-cb6f-494a-bad6-899636d56fde None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:42.446581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6de7a-cb6f-494a-bad6-899636d56fde None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:42.446884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6de7a-cb6f-494a-bad6-899636d56fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:42.467139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-12b6de7a-cb6f-494a-bad6-899636d56fde None None] MFA Rules not processed for user `8bdfee122af8429c9b8766949b781aee`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:42.468279 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-12b6de7a-cb6f-494a-bad6-899636d56fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:42.528116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 961/1913] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:44:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:51.356688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:52.521191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:56.081312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19efde-efa3-4b13-84ff-a998195b5f7e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:56.083863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-de19efde-efa3-4b13-84ff-a998195b5f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:56.164289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19efde-efa3-4b13-84ff-a998195b5f7e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:56.169990 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5ed5a-875d-48ee-8de2-185727acd8d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:56.170813 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ed5a-875d-48ee-8de2-185727acd8d9 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:56.171085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ed5a-875d-48ee-8de2-185727acd8d9 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:56.171358 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5ed5a-875d-48ee-8de2-185727acd8d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:56.173107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0ab5ed5a-875d-48ee-8de2-185727acd8d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:56.186431 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5ed5a-875d-48ee-8de2-185727acd8d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:56.188078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5ed5a-875d-48ee-8de2-185727acd8d9 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:56.198895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 953/1914] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:57.057461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e8411-f730-4b10-beef-fb630b6e646f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:57.057986 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e8411-f730-4b10-beef-fb630b6e646f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:57.057986 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e8411-f730-4b10-beef-fb630b6e646f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:57.091648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-235e8411-f730-4b10-beef-fb630b6e646f None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:57.092900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-235e8411-f730-4b10-beef-fb630b6e646f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:57.101867 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 962/1915] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:44:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:57.122394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5ed5a-875d-48ee-8de2-185727acd8d9 service placement] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:57.136772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5ed5a-875d-48ee-8de2-185727acd8d9 service placement] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:57.142393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909adb04-72e8-4acb-b111-610937f34788 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:57.143011 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909adb04-72e8-4acb-b111-610937f34788 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:57.143291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909adb04-72e8-4acb-b111-610937f34788 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:57.143695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909adb04-72e8-4acb-b111-610937f34788 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:57.155796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909adb04-72e8-4acb-b111-610937f34788 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:57.157738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909adb04-72e8-4acb-b111-610937f34788 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:57.169684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 954/1916] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:44:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:45:07.082616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:07.164771 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=38 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:41.495269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808f20c-fee1-4269-b4a9-5763c67fd2a9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:41.495813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808f20c-fee1-4269-b4a9-5763c67fd2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:41.495813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808f20c-fee1-4269-b4a9-5763c67fd2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:41.514645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a808f20c-fee1-4269-b4a9-5763c67fd2a9 None None] MFA Rules not processed for user `687126fda40f481cb5ba5d185cbfa74a`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:41.515888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a808f20c-fee1-4269-b4a9-5763c67fd2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:41.569157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 963/1917] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:45:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:47.429611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91b47e-ea2e-42a4-b20d-9e67c24d105d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:47.430545 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91b47e-ea2e-42a4-b20d-9e67c24d105d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:47.430824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91b47e-ea2e-42a4-b20d-9e67c24d105d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:47.462453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2e91b47e-ea2e-42a4-b20d-9e67c24d105d None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:47.462453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2e91b47e-ea2e-42a4-b20d-9e67c24d105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:47.480765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 955/1918] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:45:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:47.504852 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808f20c-fee1-4269-b4a9-5763c67fd2a9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:47.517701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808f20c-fee1-4269-b4a9-5763c67fd2a9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:47.523702 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218df727-18b7-481a-93d5-1a814ebaa19a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:47.524440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218df727-18b7-481a-93d5-1a814ebaa19a service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:47.525367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218df727-18b7-481a-93d5-1a814ebaa19a service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:47.526001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218df727-18b7-481a-93d5-1a814ebaa19a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:47.535744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218df727-18b7-481a-93d5-1a814ebaa19a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:47.537972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218df727-18b7-481a-93d5-1a814ebaa19a service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:47.548352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 964/1919] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:45:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:45:57.452095 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:57.544503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=31 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:46:00.411449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91b47e-ea2e-42a4-b20d-9e67c24d105d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:00.477296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91b47e-ea2e-42a4-b20d-9e67c24d105d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:00.481197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9477dc-ea29-4281-b629-52a323c8bcfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:00.481921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9477dc-ea29-4281-b629-52a323c8bcfa service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:00.482337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9477dc-ea29-4281-b629-52a323c8bcfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:00.482733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9477dc-ea29-4281-b629-52a323c8bcfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:00.492268 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9477dc-ea29-4281-b629-52a323c8bcfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:00.493823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9477dc-ea29-4281-b629-52a323c8bcfa service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:00.507097 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 956/1920] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:46:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:46:04.627806 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3af46-5c34-427b-9fe7-0b7a2b6ad63e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:04.628679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3af46-5c34-427b-9fe7-0b7a2b6ad63e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:04.628957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3af46-5c34-427b-9fe7-0b7a2b6ad63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:04.652784 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-bce3af46-5c34-427b-9fe7-0b7a2b6ad63e None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:46:04.654084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bce3af46-5c34-427b-9fe7-0b7a2b6ad63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:04.662212 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 965/1921] 10.222.0.20 () {58 vars in 967 bytes} [Mon Oct 24 13:46:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:46:04.676104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9477dc-ea29-4281-b629-52a323c8bcfa service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:04.688142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9477dc-ea29-4281-b629-52a323c8bcfa service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:04.694313 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2bc96-de38-40ac-93f5-6ec67e4a416c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:04.695518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2bc96-de38-40ac-93f5-6ec67e4a416c service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:04.695872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2bc96-de38-40ac-93f5-6ec67e4a416c service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:04.696249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2bc96-de38-40ac-93f5-6ec67e4a416c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:04.707225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2bc96-de38-40ac-93f5-6ec67e4a416c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:04.708843 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2bc96-de38-40ac-93f5-6ec67e4a416c service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:04.727987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 957/1922] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:46:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:46:06.962686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3af46-5c34-427b-9fe7-0b7a2b6ad63e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:06.964176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bce3af46-5c34-427b-9fe7-0b7a2b6ad63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:07.040849 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3af46-5c34-427b-9fe7-0b7a2b6ad63e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:07.045733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed47317-7d30-49c2-beec-3ef2a1773894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:07.046388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed47317-7d30-49c2-beec-3ef2a1773894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:07.046666 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed47317-7d30-49c2-beec-3ef2a1773894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:07.046935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed47317-7d30-49c2-beec-3ef2a1773894 admin admin] PATH_INFO: `/v3/users/8bdfee122af8429c9b8766949b781aee` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:07.048051 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed47317-7d30-49c2-beec-3ef2a1773894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bdfee122af8429c9b8766949b781aee)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:07.049939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed47317-7d30-49c2-beec-3ef2a1773894 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:07.095409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bed47317-7d30-49c2-beec-3ef2a1773894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bdfee122af8429c9b8766949b781aee'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:07.096050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bed47317-7d30-49c2-beec-3ef2a1773894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bdfee122af8429c9b8766949b781aee'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:07.108200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bed47317-7d30-49c2-beec-3ef2a1773894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bdfee122af8429c9b8766949b781aee'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:07.108834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bed47317-7d30-49c2-beec-3ef2a1773894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bdfee122af8429c9b8766949b781aee'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:07.115628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-bed47317-7d30-49c2-beec-3ef2a1773894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bdfee122af8429c9b8766949b781aee'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:07.129340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 966/1923] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:46:06 2022] DELETE /identity/v3/users/8bdfee122af8429c9b8766949b781aee => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:46:07.302702 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2bc96-de38-40ac-93f5-6ec67e4a416c service swift] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:07.317161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2bc96-de38-40ac-93f5-6ec67e4a416c service swift] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:07.374101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:07.374756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:07.375037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:07.375308 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] PATH_INFO: `/v3/projects/0bc5e5a0d29e43589d95be16c1fb464e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:07.376426 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bc5e5a0d29e43589d95be16c1fb464e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:07.378353 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:07.423976 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bc5e5a0d29e43589d95be16c1fb464e', 'request_id': 'req-8293b54d-e192-4ce0-ad81-bd24a1eea145'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:07.431841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bc5e5a0d29e43589d95be16c1fb464e', 'request_id': 'req-8293b54d-e192-4ce0-ad81-bd24a1eea145'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:07.437974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bc5e5a0d29e43589d95be16c1fb464e', 'request_id': 'req-8293b54d-e192-4ce0-ad81-bd24a1eea145'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:07.448994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] The token cache is being invalidate because project 0bc5e5a0d29e43589d95be16c1fb464e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:46:07.449276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:07.449897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8293b54d-e192-4ce0-ad81-bd24a1eea145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:07.451373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 958/1924] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:46:07 2022] DELETE /identity/v3/projects/0bc5e5a0d29e43589d95be16c1fb464e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:46:07.457381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89537f-cdc0-47d9-a454-17fb31ef7147 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:07.457674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89537f-cdc0-47d9-a454-17fb31ef7147 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:07.457949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89537f-cdc0-47d9-a454-17fb31ef7147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:07.477846 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ab89537f-cdc0-47d9-a454-17fb31ef7147 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:46:07.479100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ab89537f-cdc0-47d9-a454-17fb31ef7147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:07.537567 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 967/1925] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:46:07.545848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e13b81-36c7-4781-afea-b74b01828421 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:07.546151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e13b81-36c7-4781-afea-b74b01828421 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:07.546445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e13b81-36c7-4781-afea-b74b01828421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:07.566535 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-86e13b81-36c7-4781-afea-b74b01828421 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:46:07.567720 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-86e13b81-36c7-4781-afea-b74b01828421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:07.575548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 959/1926] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:46:07.579710 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89537f-cdc0-47d9-a454-17fb31ef7147 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:07.589550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89537f-cdc0-47d9-a454-17fb31ef7147 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:07.594059 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29566f34-a45c-4867-92dd-94865310461a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:07.594688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29566f34-a45c-4867-92dd-94865310461a admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:07.594966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29566f34-a45c-4867-92dd-94865310461a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:07.595237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29566f34-a45c-4867-92dd-94865310461a admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:07.595879 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29566f34-a45c-4867-92dd-94865310461a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:46:07.596179 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29566f34-a45c-4867-92dd-94865310461a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:07.597749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29566f34-a45c-4867-92dd-94865310461a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:07.606891 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:46:07.610476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 968/1927] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:46:07.614972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e13b81-36c7-4781-afea-b74b01828421 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:07.624701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e13b81-36c7-4781-afea-b74b01828421 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:07.629286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad557180-37f9-42c3-a484-901aad215119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:07.629878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad557180-37f9-42c3-a484-901aad215119 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:07.630156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad557180-37f9-42c3-a484-901aad215119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:07.630457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad557180-37f9-42c3-a484-901aad215119 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:07.631081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad557180-37f9-42c3-a484-901aad215119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:07.633026 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad557180-37f9-42c3-a484-901aad215119 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:07.643918 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:46:07.660580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 960/1928] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:46:07 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:46:07.665311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29566f34-a45c-4867-92dd-94865310461a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:07.675020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29566f34-a45c-4867-92dd-94865310461a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:07.731002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290e174-0435-4046-adaf-9f683903d59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:07.731617 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290e174-0435-4046-adaf-9f683903d59f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:07.731899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290e174-0435-4046-adaf-9f683903d59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:07.732167 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290e174-0435-4046-adaf-9f683903d59f admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:07.732795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290e174-0435-4046-adaf-9f683903d59f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:07.734724 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290e174-0435-4046-adaf-9f683903d59f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:07.757376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 969/1929] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:46:07 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:46:07.761474 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad557180-37f9-42c3-a484-901aad215119 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:07.771591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad557180-37f9-42c3-a484-901aad215119 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:07.776206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc75ccd-0897-48c4-b991-97dd1a17257f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:07.776793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc75ccd-0897-48c4-b991-97dd1a17257f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:07.777089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc75ccd-0897-48c4-b991-97dd1a17257f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:07.777343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc75ccd-0897-48c4-b991-97dd1a17257f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:07.777981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc75ccd-0897-48c4-b991-97dd1a17257f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:07.779594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc75ccd-0897-48c4-b991-97dd1a17257f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:07.790333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 961/1930] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:46:07.794200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290e174-0435-4046-adaf-9f683903d59f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:07.804466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290e174-0435-4046-adaf-9f683903d59f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:07.809085 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac6fbe-0433-489e-b30b-622ca21fae3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:07.809743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac6fbe-0433-489e-b30b-622ca21fae3d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:07.809911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac6fbe-0433-489e-b30b-622ca21fae3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:07.810191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac6fbe-0433-489e-b30b-622ca21fae3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:07.810834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac6fbe-0433-489e-b30b-622ca21fae3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:07.812406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac6fbe-0433-489e-b30b-622ca21fae3d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:07.823107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 970/1931] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:46:07.827435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc75ccd-0897-48c4-b991-97dd1a17257f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:07.837603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc75ccd-0897-48c4-b991-97dd1a17257f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:07.842951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2438bb-9327-4933-9139-8d46445dea5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:07.843599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2438bb-9327-4933-9139-8d46445dea5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:07.843998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2438bb-9327-4933-9139-8d46445dea5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:07.844179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2438bb-9327-4933-9139-8d46445dea5b admin admin] PATH_INFO: `/v3/projects/e1a52980f96f4dbdbe326c83bdf5fafa/users/3690c8f3b498452da0514eb9c0788537/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:07.859865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2438bb-9327-4933-9139-8d46445dea5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a52980f96f4dbdbe326c83bdf5fafa, user_id=3690c8f3b498452da0514eb9c0788537, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:07.862531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2438bb-9327-4933-9139-8d46445dea5b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:07.875797 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 962/1932] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:46:07 2022] PUT /identity/v3/projects/e1a52980f96f4dbdbe326c83bdf5fafa/users/3690c8f3b498452da0514eb9c0788537/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:46:07.944529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60401782-eccf-49f3-b518-3cabb7d47540 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:07.944931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60401782-eccf-49f3-b518-3cabb7d47540 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:07.945213 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60401782-eccf-49f3-b518-3cabb7d47540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:07.973428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-60401782-eccf-49f3-b518-3cabb7d47540 None None] MFA Rules not processed for user `3690c8f3b498452da0514eb9c0788537`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:46:07.974685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-60401782-eccf-49f3-b518-3cabb7d47540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:08.049805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 971/1933] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:46:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:46:08.059205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2438bb-9327-4933-9139-8d46445dea5b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:08.061045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7e2438bb-9327-4933-9139-8d46445dea5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:08.120081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2438bb-9327-4933-9139-8d46445dea5b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:08.123723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3326a9-6fc9-4eae-a810-36bfb9abc189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:08.124320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3326a9-6fc9-4eae-a810-36bfb9abc189 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:08.124594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3326a9-6fc9-4eae-a810-36bfb9abc189 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:08.124860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3326a9-6fc9-4eae-a810-36bfb9abc189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:08.133133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3326a9-6fc9-4eae-a810-36bfb9abc189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:08.134569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3326a9-6fc9-4eae-a810-36bfb9abc189 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:08.143196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 963/1934] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:46:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:46:18.039484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:46:18.139234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:46:18.638757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd410-6a9e-4baf-9100-04af1b7aea43 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:18.639044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd410-6a9e-4baf-9100-04af1b7aea43 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:18.639318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd410-6a9e-4baf-9100-04af1b7aea43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:18.660691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-421bd410-6a9e-4baf-9100-04af1b7aea43 None None] MFA Rules not processed for user `1461a94dd8ef48f79a0d8642a1bc1e58`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:46:18.661882 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-421bd410-6a9e-4baf-9100-04af1b7aea43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:18.716232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 972/1935] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:46:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:46:28.708799 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=25 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:46:38.845702 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3326a9-6fc9-4eae-a810-36bfb9abc189 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:38.907612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3326a9-6fc9-4eae-a810-36bfb9abc189 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:38.912094 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:38.912712 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:38.912988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:38.913264 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] PATH_INFO: `/v3/users/3690c8f3b498452da0514eb9c0788537` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:38.914433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3690c8f3b498452da0514eb9c0788537)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:38.916430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:38.971473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3690c8f3b498452da0514eb9c0788537'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:38.972166 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3690c8f3b498452da0514eb9c0788537'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:38.972777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3690c8f3b498452da0514eb9c0788537'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:38.979986 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3690c8f3b498452da0514eb9c0788537'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:38.991881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3690c8f3b498452da0514eb9c0788537'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:39.005196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 964/1936] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:46:38 2022] DELETE /identity/v3/users/3690c8f3b498452da0514eb9c0788537 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:46:39.010374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ca680-7280-46f7-95f4-9acb6745ee44 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:39.010877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ca680-7280-46f7-95f4-9acb6745ee44 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:39.010982 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ca680-7280-46f7-95f4-9acb6745ee44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:39.029517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-753ca680-7280-46f7-95f4-9acb6745ee44 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:46:39.030726 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-753ca680-7280-46f7-95f4-9acb6745ee44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:39.088720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 973/1937] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:46:39.098325 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:39.099708 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:39.156527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305d525-beff-425a-a401-a505c33bc41c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:39.160401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75d8b0-93a3-4853-b4db-0c4d3d725829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:39.160992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75d8b0-93a3-4853-b4db-0c4d3d725829 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:39.161270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75d8b0-93a3-4853-b4db-0c4d3d725829 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:39.161537 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75d8b0-93a3-4853-b4db-0c4d3d725829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:39.171594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75d8b0-93a3-4853-b4db-0c4d3d725829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:39.173040 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75d8b0-93a3-4853-b4db-0c4d3d725829 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:39.183420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 965/1938] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:46:39.464250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ca680-7280-46f7-95f4-9acb6745ee44 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:39.465860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-753ca680-7280-46f7-95f4-9acb6745ee44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:39.480187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ca680-7280-46f7-95f4-9acb6745ee44 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:39.484766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:39.485398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:39.485672 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:39.485939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] PATH_INFO: `/v3/projects/e1a52980f96f4dbdbe326c83bdf5fafa` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:39.487161 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a52980f96f4dbdbe326c83bdf5fafa)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:39.489169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:39.536056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a52980f96f4dbdbe326c83bdf5fafa', 'request_id': 'req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:39.542338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a52980f96f4dbdbe326c83bdf5fafa', 'request_id': 'req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:39.549656 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a52980f96f4dbdbe326c83bdf5fafa', 'request_id': 'req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:39.560861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] The token cache is being invalidate because project e1a52980f96f4dbdbe326c83bdf5fafa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:46:39.561189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:39.561850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:46:39.563370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 974/1939] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:46:39 2022] DELETE /identity/v3/projects/e1a52980f96f4dbdbe326c83bdf5fafa => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:46:44.682783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75d8b0-93a3-4853-b4db-0c4d3d725829 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:44.684257 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ba75d8b0-93a3-4853-b4db-0c4d3d725829 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:44.695260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75d8b0-93a3-4853-b4db-0c4d3d725829 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:44.713373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:46:44.746091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504d2b9-a063-4b0e-9daf-a16eb3804d7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:44.746768 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504d2b9-a063-4b0e-9daf-a16eb3804d7e service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:44.747049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504d2b9-a063-4b0e-9daf-a16eb3804d7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:44.747319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504d2b9-a063-4b0e-9daf-a16eb3804d7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:44.749064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3504d2b9-a063-4b0e-9daf-a16eb3804d7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:44.819225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504d2b9-a063-4b0e-9daf-a16eb3804d7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:44.820667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504d2b9-a063-4b0e-9daf-a16eb3804d7e service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:44.831405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 966/1940] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:46:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:46:47.511108 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:46:47.512511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:47.569645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c31242-12a9-4c8b-85a8-c3b54c393dcb admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:46:47.573389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40eab02-33f0-449c-9abe-5d72dfbcc558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:46:47.574002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eab02-33f0-449c-9abe-5d72dfbcc558 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:46:47.574319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eab02-33f0-449c-9abe-5d72dfbcc558 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:46:47.574591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eab02-33f0-449c-9abe-5d72dfbcc558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:46:47.576320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b40eab02-33f0-449c-9abe-5d72dfbcc558 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:46:47.589655 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40eab02-33f0-449c-9abe-5d72dfbcc558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:46:47.591106 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40eab02-33f0-449c-9abe-5d72dfbcc558 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:46:47.602838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 975/1941] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:46:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:46:54.826568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=44 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:46:57.597107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:47:20.849420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb35a5-3d15-4c5e-8395-3589f6ecd556 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:20.850388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb35a5-3d15-4c5e-8395-3589f6ecd556 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:20.850686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb35a5-3d15-4c5e-8395-3589f6ecd556 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:20.852366 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 967/1942] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:47:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:47:20.879738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9eaf5c-45e4-4703-9035-6df32ca44313 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:20.880249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9eaf5c-45e4-4703-9035-6df32ca44313 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:20.880249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9eaf5c-45e4-4703-9035-6df32ca44313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:20.899680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1a9eaf5c-45e4-4703-9035-6df32ca44313 None None] MFA Rules not processed for user `e0ce1753e22d4952ab5d4348b0cffc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:47:20.900872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1a9eaf5c-45e4-4703-9035-6df32ca44313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:20.914906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb35a5-3d15-4c5e-8395-3589f6ecd556 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:20.917017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-50fb35a5-3d15-4c5e-8395-3589f6ecd556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:20.970258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 976/1943] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:47:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:47:20.981505 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9eaf5c-45e4-4703-9035-6df32ca44313 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:20.982870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1a9eaf5c-45e4-4703-9035-6df32ca44313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:20.991035 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb35a5-3d15-4c5e-8395-3589f6ecd556 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:20.995504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:20.995755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9eaf5c-45e4-4703-9035-6df32ca44313 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:20.996218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:20.996492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:20.996756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] PATH_INFO: `/v3/users/687126fda40f481cb5ba5d185cbfa74a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:20.997953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] RBAC: Authorizing `identity:delete_user(user_id=687126fda40f481cb5ba5d185cbfa74a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:20.999880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:21.000919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829e0e3-909e-4923-a3e3-d2fc2b1aab85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:21.001709 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829e0e3-909e-4923-a3e3-d2fc2b1aab85 service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:21.001987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829e0e3-909e-4923-a3e3-d2fc2b1aab85 service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:21.002288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829e0e3-909e-4923-a3e3-d2fc2b1aab85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:21.012298 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829e0e3-909e-4923-a3e3-d2fc2b1aab85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:21.013895 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829e0e3-909e-4923-a3e3-d2fc2b1aab85 service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:21.025124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 977/1944] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:47:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:47:21.049321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '687126fda40f481cb5ba5d185cbfa74a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:47:21.049991 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '687126fda40f481cb5ba5d185cbfa74a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:47:21.050651 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '687126fda40f481cb5ba5d185cbfa74a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:47:21.058084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '687126fda40f481cb5ba5d185cbfa74a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:47:21.070007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '687126fda40f481cb5ba5d185cbfa74a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:47:21.083666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 968/1945] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:47:20 2022] DELETE /identity/v3/users/687126fda40f481cb5ba5d185cbfa74a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:47:21.264761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829e0e3-909e-4923-a3e3-d2fc2b1aab85 service placement] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:21.275405 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829e0e3-909e-4923-a3e3-d2fc2b1aab85 service placement] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:21.338189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:21.338945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:21.339366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:21.339645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] PATH_INFO: `/v3/projects/b8ee40c2e5304b67b8814d34f3fe97b3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:21.340777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ee40c2e5304b67b8814d34f3fe97b3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:21.342698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:21.389660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ee40c2e5304b67b8814d34f3fe97b3', 'request_id': 'req-24188e2a-f795-422f-833b-531d92d85d63'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:47:21.395287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ee40c2e5304b67b8814d34f3fe97b3', 'request_id': 'req-24188e2a-f795-422f-833b-531d92d85d63'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:47:21.402777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ee40c2e5304b67b8814d34f3fe97b3', 'request_id': 'req-24188e2a-f795-422f-833b-531d92d85d63'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:47:21.414011 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] The token cache is being invalidate because project b8ee40c2e5304b67b8814d34f3fe97b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:47:21.414330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:47:21.414948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-24188e2a-f795-422f-833b-531d92d85d63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:47:21.416450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 978/1946] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:47:21 2022] DELETE /identity/v3/projects/b8ee40c2e5304b67b8814d34f3fe97b3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:47:29.332013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:29.333870 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:29.408122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f779ed-fd67-481a-87e6-eb055302464e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:29.421412 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:47:29.473385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c8081-2544-474e-acd5-3d955e653e3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:29.473917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c8081-2544-474e-acd5-3d955e653e3a service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:29.474221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c8081-2544-474e-acd5-3d955e653e3a service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:29.474497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c8081-2544-474e-acd5-3d955e653e3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:29.476232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3c4c8081-2544-474e-acd5-3d955e653e3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:29.488425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c8081-2544-474e-acd5-3d955e653e3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:29.490007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c8081-2544-474e-acd5-3d955e653e3a service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:29.501140 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 969/1947] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:47:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:47:31.411074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:47:33.895984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa0961-08cd-4c03-962d-255fed09feaa None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:33.896579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa0961-08cd-4c03-962d-255fed09feaa None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:33.896579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa0961-08cd-4c03-962d-255fed09feaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:33.918499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-86fa0961-08cd-4c03-962d-255fed09feaa None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:47:33.919631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-86fa0961-08cd-4c03-962d-255fed09feaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:33.973826 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 979/1948] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:47:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:47:34.024168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c8081-2544-474e-acd5-3d955e653e3a service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:34.025632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3c4c8081-2544-474e-acd5-3d955e653e3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:34.095725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c8081-2544-474e-acd5-3d955e653e3a service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:34.100732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3031a6e-5b33-4019-aa6b-1068db1bc272 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:34.101333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3031a6e-5b33-4019-aa6b-1068db1bc272 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:34.101612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3031a6e-5b33-4019-aa6b-1068db1bc272 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:34.101884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3031a6e-5b33-4019-aa6b-1068db1bc272 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:34.110436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3031a6e-5b33-4019-aa6b-1068db1bc272 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:34.112012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3031a6e-5b33-4019-aa6b-1068db1bc272 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:34.119943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 970/1949] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:47:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:47:43.967673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=25 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:47:44.116699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=58 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:47:47.148235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa0961-08cd-4c03-962d-255fed09feaa None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:47.149660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-86fa0961-08cd-4c03-962d-255fed09feaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:47.207840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa0961-08cd-4c03-962d-255fed09feaa None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:47.211669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce00d4d-0b4d-4ad2-91c2-59c91976af07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:47.212272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce00d4d-0b4d-4ad2-91c2-59c91976af07 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:47.212549 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce00d4d-0b4d-4ad2-91c2-59c91976af07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:47.212814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce00d4d-0b4d-4ad2-91c2-59c91976af07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:47.214573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fce00d4d-0b4d-4ad2-91c2-59c91976af07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:47.224124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce00d4d-0b4d-4ad2-91c2-59c91976af07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:47.225533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce00d4d-0b4d-4ad2-91c2-59c91976af07 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:47.233822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 980/1950] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:47:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:47:50.516779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3031a6e-5b33-4019-aa6b-1068db1bc272 service swift] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:50.525890 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3031a6e-5b33-4019-aa6b-1068db1bc272 service swift] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:50.529412 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae018c20-61ed-42a4-810b-51c1198da47b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:50.529996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae018c20-61ed-42a4-810b-51c1198da47b service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:50.530428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae018c20-61ed-42a4-810b-51c1198da47b service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:50.530703 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae018c20-61ed-42a4-810b-51c1198da47b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:50.532694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ae018c20-61ed-42a4-810b-51c1198da47b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:50.541843 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae018c20-61ed-42a4-810b-51c1198da47b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:50.543282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae018c20-61ed-42a4-810b-51c1198da47b service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:50.550978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 971/1951] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:47:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:47:50.675064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa68fb2-d409-4862-8ce3-fb1ae9ef0b03 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:50.675635 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa68fb2-d409-4862-8ce3-fb1ae9ef0b03 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:50.675747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa68fb2-d409-4862-8ce3-fb1ae9ef0b03 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:50.677033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 981/1952] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:47:50.684067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92908f-065b-42fb-b2dc-94bb57646134 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:50.684348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92908f-065b-42fb-b2dc-94bb57646134 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:50.684645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92908f-065b-42fb-b2dc-94bb57646134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:50.706519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3e92908f-065b-42fb-b2dc-94bb57646134 None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:47:50.707729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3e92908f-065b-42fb-b2dc-94bb57646134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:50.716957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 972/1953] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:47:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:47:50.728920 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa68fb2-d409-4862-8ce3-fb1ae9ef0b03 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:50.730612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-baa68fb2-d409-4862-8ce3-fb1ae9ef0b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:50.791153 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa68fb2-d409-4862-8ce3-fb1ae9ef0b03 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:50.794725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d86ac4-8e0d-47bf-a225-af474192a86e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07f8443d5594f47bc81e9f9dc9ce43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:50.795361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d86ac4-8e0d-47bf-a225-af474192a86e service glance] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:50.795641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d86ac4-8e0d-47bf-a225-af474192a86e service glance] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:50.796491 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d86ac4-8e0d-47bf-a225-af474192a86e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:50.806875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d86ac4-8e0d-47bf-a225-af474192a86e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:50.808287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d86ac4-8e0d-47bf-a225-af474192a86e service glance] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:50.819727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 982/1954] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:47:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:47:50.889773 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4935985-7130-41a5-8a59-3a5761c3d870 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:50.890055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4935985-7130-41a5-8a59-3a5761c3d870 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:50.890352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4935985-7130-41a5-8a59-3a5761c3d870 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:50.891700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 973/1955] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:47:50.896370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525a849-09e9-4de7-8916-4c6a33990923 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:50.896650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525a849-09e9-4de7-8916-4c6a33990923 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:50.896963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525a849-09e9-4de7-8916-4c6a33990923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:50.916497 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-8525a849-09e9-4de7-8916-4c6a33990923 None None] MFA Rules not processed for user `5b8479c571ec449fbfc306d25fbf7fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:47:50.917671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8525a849-09e9-4de7-8916-4c6a33990923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:50.924761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 983/1956] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:47:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:47:50.936259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4935985-7130-41a5-8a59-3a5761c3d870 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:50.948492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4935985-7130-41a5-8a59-3a5761c3d870 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:50.951899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9091588-fee3-43d0-9b6a-ed187fdd9b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:50.952606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9091588-fee3-43d0-9b6a-ed187fdd9b12 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:50.952884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9091588-fee3-43d0-9b6a-ed187fdd9b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:50.953148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9091588-fee3-43d0-9b6a-ed187fdd9b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:50.960048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9091588-fee3-43d0-9b6a-ed187fdd9b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:50.961446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9091588-fee3-43d0-9b6a-ed187fdd9b12 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:50.969051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 974/1957] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:47:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:47:51.050037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3700-6b5a-443f-bcd6-8bc8087fc4ff None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:51.050356 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3700-6b5a-443f-bcd6-8bc8087fc4ff None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:51.050627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a3700-6b5a-443f-bcd6-8bc8087fc4ff None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:51.052009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 984/1958] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:47:51.057559 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60be561-9af2-4165-9cb7-6a299a54e418 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:51.057873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60be561-9af2-4165-9cb7-6a299a54e418 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:51.058140 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60be561-9af2-4165-9cb7-6a299a54e418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:51.078047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d60be561-9af2-4165-9cb7-6a299a54e418 None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:47:51.079273 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d60be561-9af2-4165-9cb7-6a299a54e418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:51.088369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 975/1959] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:47:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:47:51.098659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a3700-6b5a-443f-bcd6-8bc8087fc4ff None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:51.110362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a3700-6b5a-443f-bcd6-8bc8087fc4ff None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:51.113906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42437b1e-6820-4c62-ab94-1e10fbcf3ef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:51.114519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42437b1e-6820-4c62-ab94-1e10fbcf3ef6 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:51.114795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42437b1e-6820-4c62-ab94-1e10fbcf3ef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:51.115060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42437b1e-6820-4c62-ab94-1e10fbcf3ef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:51.125637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42437b1e-6820-4c62-ab94-1e10fbcf3ef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:51.127080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42437b1e-6820-4c62-ab94-1e10fbcf3ef6 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:51.137393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 985/1960] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:47:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:47:51.200534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789a13e-7490-48ee-9eb7-78d9580c69e3 None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:51.200765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789a13e-7490-48ee-9eb7-78d9580c69e3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:51.201038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789a13e-7490-48ee-9eb7-78d9580c69e3 None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:51.202447 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 976/1961] 10.222.0.20 () {54 vars in 961 bytes} [Mon Oct 24 13:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:47:51.207652 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379c9c-6e3d-41e2-a231-ce30d59a9d8d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:51.207937 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379c9c-6e3d-41e2-a231-ce30d59a9d8d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:51.208204 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379c9c-6e3d-41e2-a231-ce30d59a9d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:51.228017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7c379c9c-6e3d-41e2-a231-ce30d59a9d8d None None] MFA Rules not processed for user `e0ce1753e22d4952ab5d4348b0cffc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:47:51.229195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7c379c9c-6e3d-41e2-a231-ce30d59a9d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:51.291938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 986/1962] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Oct 24 13:47:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:47:51.299949 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789a13e-7490-48ee-9eb7-78d9580c69e3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:51.313998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789a13e-7490-48ee-9eb7-78d9580c69e3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:51.319207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badee05-eb1e-4c2a-9eee-4f3992ec50de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:51.319947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badee05-eb1e-4c2a-9eee-4f3992ec50de service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:51.320344 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badee05-eb1e-4c2a-9eee-4f3992ec50de service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:51.320614 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badee05-eb1e-4c2a-9eee-4f3992ec50de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:51.322453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4badee05-eb1e-4c2a-9eee-4f3992ec50de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:51.334640 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badee05-eb1e-4c2a-9eee-4f3992ec50de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:51.336209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badee05-eb1e-4c2a-9eee-4f3992ec50de service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:51.346338 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 977/1963] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:47:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:47:53.767675 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107626a0-aeed-44ee-a387-b1c1f9bf4594 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:53.768133 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107626a0-aeed-44ee-a387-b1c1f9bf4594 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:53.768283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107626a0-aeed-44ee-a387-b1c1f9bf4594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:53.787835 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-107626a0-aeed-44ee-a387-b1c1f9bf4594 None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:47:53.789005 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-107626a0-aeed-44ee-a387-b1c1f9bf4594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:53.796026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 987/1964] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:47:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:47:53.807961 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badee05-eb1e-4c2a-9eee-4f3992ec50de service placement] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:53.819142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badee05-eb1e-4c2a-9eee-4f3992ec50de service placement] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:53.823985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aedb4d-ad61-4c00-ac12-d1fc06ea91ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:53.824567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aedb4d-ad61-4c00-ac12-d1fc06ea91ea service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:53.824843 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aedb4d-ad61-4c00-ac12-d1fc06ea91ea service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:53.825109 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aedb4d-ad61-4c00-ac12-d1fc06ea91ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:53.833463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aedb4d-ad61-4c00-ac12-d1fc06ea91ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:53.835044 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aedb4d-ad61-4c00-ac12-d1fc06ea91ea service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:53.842852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 978/1965] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:47:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:47:57.868451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107626a0-aeed-44ee-a387-b1c1f9bf4594 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:47:57.869908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-107626a0-aeed-44ee-a387-b1c1f9bf4594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:57.889740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107626a0-aeed-44ee-a387-b1c1f9bf4594 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:47:57.895987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b065b22-1bf1-4af5-91e8-c5ccaaefda6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:47:57.896616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b065b22-1bf1-4af5-91e8-c5ccaaefda6f service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:47:57.896892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b065b22-1bf1-4af5-91e8-c5ccaaefda6f service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:47:57.897160 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b065b22-1bf1-4af5-91e8-c5ccaaefda6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:47:57.898908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2b065b22-1bf1-4af5-91e8-c5ccaaefda6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:47:57.913647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:47:57.965082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b065b22-1bf1-4af5-91e8-c5ccaaefda6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:47:57.966732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b065b22-1bf1-4af5-91e8-c5ccaaefda6f service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:47:57.976518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 988/1966] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:47:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:48:03.839661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=30 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:48:05.674374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aedb4d-ad61-4c00-ac12-d1fc06ea91ea service swift] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:48:05.675785 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-52aedb4d-ad61-4c00-ac12-d1fc06ea91ea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:48:05.691273 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aedb4d-ad61-4c00-ac12-d1fc06ea91ea service swift] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:48:05.696666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8afa7c-be1c-4504-8ff1-c9abf84b1013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:48:05.697253 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8afa7c-be1c-4504-8ff1-c9abf84b1013 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:48:05.697530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8afa7c-be1c-4504-8ff1-c9abf84b1013 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:48:05.697794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8afa7c-be1c-4504-8ff1-c9abf84b1013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:48:05.699473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5e8afa7c-be1c-4504-8ff1-c9abf84b1013 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:48:05.756556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8afa7c-be1c-4504-8ff1-c9abf84b1013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:48:05.758160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8afa7c-be1c-4504-8ff1-c9abf84b1013 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:48:05.767993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 979/1967] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:48:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:48:07.972143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=21 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:48:15.763003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=28 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:48:24.184184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cab597-1204-4077-ac8b-b6fa80edabb4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:48:24.184636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cab597-1204-4077-ac8b-b6fa80edabb4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:48:24.184728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cab597-1204-4077-ac8b-b6fa80edabb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:48:24.206148 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-86cab597-1204-4077-ac8b-b6fa80edabb4 None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:48:24.207673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-86cab597-1204-4077-ac8b-b6fa80edabb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:48:24.215522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 989/1968] 10.222.0.20 () {58 vars in 967 bytes} [Mon Oct 24 13:48:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:48:24.232634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8afa7c-be1c-4504-8ff1-c9abf84b1013 service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:48:24.243661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8afa7c-be1c-4504-8ff1-c9abf84b1013 service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:48:24.248528 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41026414-abc0-4842-b3c2-6e56fc24d683 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:48:24.249114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41026414-abc0-4842-b3c2-6e56fc24d683 service swift] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:48:24.249390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41026414-abc0-4842-b3c2-6e56fc24d683 service swift] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:48:24.249656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41026414-abc0-4842-b3c2-6e56fc24d683 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:48:24.257574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41026414-abc0-4842-b3c2-6e56fc24d683 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:48:24.259181 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41026414-abc0-4842-b3c2-6e56fc24d683 service swift] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:48:24.267536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 980/1969] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:48:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:48:34.201461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:48:34.263861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=6 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:48:34.299359 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cab597-1204-4077-ac8b-b6fa80edabb4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:48:34.311972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cab597-1204-4077-ac8b-b6fa80edabb4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:48:34.319138 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb838fa-c1c1-40cf-9e00-9c21735427f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:48:34.319842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb838fa-c1c1-40cf-9e00-9c21735427f8 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:48:34.320191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb838fa-c1c1-40cf-9e00-9c21735427f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:48:34.320524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb838fa-c1c1-40cf-9e00-9c21735427f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:48:34.322366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ebb838fa-c1c1-40cf-9e00-9c21735427f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:48:34.380919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb838fa-c1c1-40cf-9e00-9c21735427f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:48:34.382528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb838fa-c1c1-40cf-9e00-9c21735427f8 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:48:34.391546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 990/1970] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:48:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:48:35.262799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca8ad-bc16-4922-9223-a43eefd3a6c6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:48:35.263088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca8ad-bc16-4922-9223-a43eefd3a6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:48:35.263364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca8ad-bc16-4922-9223-a43eefd3a6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:48:35.283013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-756ca8ad-bc16-4922-9223-a43eefd3a6c6 None None] MFA Rules not processed for user `be793f4685f94beba74adedbc02ee50a`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:48:35.284211 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-756ca8ad-bc16-4922-9223-a43eefd3a6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:48:35.291126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 981/1971] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:48:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:48:37.734326 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb838fa-c1c1-40cf-9e00-9c21735427f8 service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:48:37.745843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb838fa-c1c1-40cf-9e00-9c21735427f8 service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:48:37.751582 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d36b8-6d26-4dac-a732-ad6c808e70ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:48:37.752847 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d36b8-6d26-4dac-a732-ad6c808e70ad service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:48:37.753118 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d36b8-6d26-4dac-a732-ad6c808e70ad service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:48:37.753591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d36b8-6d26-4dac-a732-ad6c808e70ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:48:37.755344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e18d36b8-6d26-4dac-a732-ad6c808e70ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:48:37.768263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d36b8-6d26-4dac-a732-ad6c808e70ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:48:37.769830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d36b8-6d26-4dac-a732-ad6c808e70ad service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:48:37.780853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 991/1972] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:48:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:48:41.410921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ca8ad-bc16-4922-9223-a43eefd3a6c6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:48:41.425460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ca8ad-bc16-4922-9223-a43eefd3a6c6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:48:41.431027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc55eb0-9062-4476-878a-75427d1c6656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:48:41.431640 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc55eb0-9062-4476-878a-75427d1c6656 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:48:41.431916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc55eb0-9062-4476-878a-75427d1c6656 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:48:41.432192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc55eb0-9062-4476-878a-75427d1c6656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:48:41.433927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6bc55eb0-9062-4476-878a-75427d1c6656 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:48:41.446514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc55eb0-9062-4476-878a-75427d1c6656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:48:41.448090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc55eb0-9062-4476-878a-75427d1c6656 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:48:41.468056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 982/1973] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:48:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:48:47.776012 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=34 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:48:48.425221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53db72b-4065-42bb-9dce-3e2a1451f7bf None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:48:48.425504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53db72b-4065-42bb-9dce-3e2a1451f7bf None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:48:48.425780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53db72b-4065-42bb-9dce-3e2a1451f7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:48:48.445175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a53db72b-4065-42bb-9dce-3e2a1451f7bf None None] MFA Rules not processed for user `1461a94dd8ef48f79a0d8642a1bc1e58`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:48:48.446405 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a53db72b-4065-42bb-9dce-3e2a1451f7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:48:48.453353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 992/1974] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:48:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:48:51.462951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=9 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:48:58.438999 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:49:33.108494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc55eb0-9062-4476-878a-75427d1c6656 service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:49:33.120911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc55eb0-9062-4476-878a-75427d1c6656 service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:49:33.127108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28dd04-ff1b-4d25-9df8-04786475a013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:49:33.127291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53db72b-4065-42bb-9dce-3e2a1451f7bf None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:49:33.128351 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28dd04-ff1b-4d25-9df8-04786475a013 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:33.130017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28dd04-ff1b-4d25-9df8-04786475a013 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:33.130346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28dd04-ff1b-4d25-9df8-04786475a013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:33.132259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4d28dd04-ff1b-4d25-9df8-04786475a013 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:33.141427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53db72b-4065-42bb-9dce-3e2a1451f7bf None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:49:33.147772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae60e-a1f0-4f01-bb37-0aeb3253b76c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:49:33.148384 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae60e-a1f0-4f01-bb37-0aeb3253b76c service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:33.148664 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae60e-a1f0-4f01-bb37-0aeb3253b76c service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:33.148935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae60e-a1f0-4f01-bb37-0aeb3253b76c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:33.150780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f2bae60e-a1f0-4f01-bb37-0aeb3253b76c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:33.193685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28dd04-ff1b-4d25-9df8-04786475a013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:49:33.195213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28dd04-ff1b-4d25-9df8-04786475a013 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:49:33.205428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 983/1975] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:49:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:49:33.212044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bae60e-a1f0-4f01-bb37-0aeb3253b76c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:49:33.213626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bae60e-a1f0-4f01-bb37-0aeb3253b76c service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:49:33.223216 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 993/1976] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:49:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:49:43.200974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=28 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:49:43.219047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=28 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:49:43.571216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21551e-edbf-4bbd-8cde-bb5afa54ea4c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:43.571828 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21551e-edbf-4bbd-8cde-bb5afa54ea4c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:43.572431 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21551e-edbf-4bbd-8cde-bb5afa54ea4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:43.599093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2d21551e-edbf-4bbd-8cde-bb5afa54ea4c None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:49:43.600429 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2d21551e-edbf-4bbd-8cde-bb5afa54ea4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:43.608788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 984/1977] 10.222.0.20 () {58 vars in 967 bytes} [Mon Oct 24 13:49:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:49:43.686723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae60e-a1f0-4f01-bb37-0aeb3253b76c service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:49:43.730013 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae60e-a1f0-4f01-bb37-0aeb3253b76c service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:49:43.735630 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5859681-e470-4aa4-a052-ca522f739b33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:49:43.736463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5859681-e470-4aa4-a052-ca522f739b33 service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:43.736909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5859681-e470-4aa4-a052-ca522f739b33 service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:43.737342 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5859681-e470-4aa4-a052-ca522f739b33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:43.746961 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5859681-e470-4aa4-a052-ca522f739b33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:49:43.748782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5859681-e470-4aa4-a052-ca522f739b33 service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:49:43.760535 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 994/1978] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:49:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:49:44.263402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21551e-edbf-4bbd-8cde-bb5afa54ea4c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:49:44.275740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21551e-edbf-4bbd-8cde-bb5afa54ea4c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:49:44.280243 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda9fa6-9ab2-433c-be23-cb0d331bf31d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:49:44.281121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda9fa6-9ab2-433c-be23-cb0d331bf31d service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:44.281578 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda9fa6-9ab2-433c-be23-cb0d331bf31d service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:44.282023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda9fa6-9ab2-433c-be23-cb0d331bf31d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:44.284169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-adda9fa6-9ab2-433c-be23-cb0d331bf31d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:44.354193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda9fa6-9ab2-433c-be23-cb0d331bf31d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:49:44.355635 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda9fa6-9ab2-433c-be23-cb0d331bf31d service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:49:44.366244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 985/1979] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:49:44.368607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75954e9c-9d7e-4772-90a8-577332d1559e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:44.369079 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75954e9c-9d7e-4772-90a8-577332d1559e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:44.369193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75954e9c-9d7e-4772-90a8-577332d1559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:44.388300 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-75954e9c-9d7e-4772-90a8-577332d1559e None None] MFA Rules not processed for user `16228db2ddd147969482871b600ac797`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:49:44.389576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-75954e9c-9d7e-4772-90a8-577332d1559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:44.397059 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 995/1980] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:49:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:49:44.771123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda9fa6-9ab2-433c-be23-cb0d331bf31d service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:49:44.772614 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-adda9fa6-9ab2-433c-be23-cb0d331bf31d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:44.784409 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda9fa6-9ab2-433c-be23-cb0d331bf31d service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:49:44.788836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b731018-62d7-49c2-b30f-bd3c37040430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:49:44.789512 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b731018-62d7-49c2-b30f-bd3c37040430 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:44.789822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b731018-62d7-49c2-b30f-bd3c37040430 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:44.790451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b731018-62d7-49c2-b30f-bd3c37040430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:44.792625 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7b731018-62d7-49c2-b30f-bd3c37040430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:44.804332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b731018-62d7-49c2-b30f-bd3c37040430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:49:44.805794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b731018-62d7-49c2-b30f-bd3c37040430 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:49:44.815703 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 986/1981] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:49:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:49:47.618437 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529978f1-1c87-4623-9b5e-eeb464a5b95d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:47.618909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529978f1-1c87-4623-9b5e-eeb464a5b95d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:47.619049 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529978f1-1c87-4623-9b5e-eeb464a5b95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:47.655495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-529978f1-1c87-4623-9b5e-eeb464a5b95d None None] MFA Rules not processed for user `1ea8feefb0614972898c1b1ad28077a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:49:47.657092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-529978f1-1c87-4623-9b5e-eeb464a5b95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:47.687712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 996/1982] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:49:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:49:48.988372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b731018-62d7-49c2-b30f-bd3c37040430 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:49:49.091902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b731018-62d7-49c2-b30f-bd3c37040430 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:49:49.100738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadf8e3-abb1-44c8-b22a-e4ff676f846c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:49:49.101366 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf8e3-abb1-44c8-b22a-e4ff676f846c service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:49.101652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf8e3-abb1-44c8-b22a-e4ff676f846c service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:49.101927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf8e3-abb1-44c8-b22a-e4ff676f846c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:49.104514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8aadf8e3-abb1-44c8-b22a-e4ff676f846c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:49.145357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadf8e3-abb1-44c8-b22a-e4ff676f846c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:49:49.148874 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadf8e3-abb1-44c8-b22a-e4ff676f846c service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:49:49.159535 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 987/1983] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:49:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:49:56.287135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529978f1-1c87-4623-9b5e-eeb464a5b95d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:49:56.288704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-529978f1-1c87-4623-9b5e-eeb464a5b95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:56.343019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529978f1-1c87-4623-9b5e-eeb464a5b95d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:49:56.347696 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:49:56.349035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:56.349320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:56.349589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] PATH_INFO: `/v3/users/6e926bb8de284795bb6ebbda2dccc266` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:56.350759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e926bb8de284795bb6ebbda2dccc266)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:49:56.352690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:49:56.402765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e926bb8de284795bb6ebbda2dccc266'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.403330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e926bb8de284795bb6ebbda2dccc266'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.416376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e926bb8de284795bb6ebbda2dccc266'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.417284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e926bb8de284795bb6ebbda2dccc266'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.427159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e926bb8de284795bb6ebbda2dccc266'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.441402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 997/1984] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:49:56 2022] DELETE /identity/v3/users/6e926bb8de284795bb6ebbda2dccc266 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:49:56.637896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadf8e3-abb1-44c8-b22a-e4ff676f846c service placement] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:49:56.649926 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadf8e3-abb1-44c8-b22a-e4ff676f846c service placement] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:49:56.713323 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:49:56.714028 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:56.714337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:56.714604 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] PATH_INFO: `/v3/projects/5243c296c8cb4467bfd269e4622e1adb` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:56.715718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5243c296c8cb4467bfd269e4622e1adb)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:49:56.717630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:49:56.738809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:49:56.771657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5243c296c8cb4467bfd269e4622e1adb', 'request_id': 'req-9b505725-d537-4c29-b7e1-707a0e292045'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.779232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5243c296c8cb4467bfd269e4622e1adb', 'request_id': 'req-9b505725-d537-4c29-b7e1-707a0e292045'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.785686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5243c296c8cb4467bfd269e4622e1adb', 'request_id': 'req-9b505725-d537-4c29-b7e1-707a0e292045'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.795901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] The token cache is being invalidate because project 5243c296c8cb4467bfd269e4622e1adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:49:56.796187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.796811 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.798331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 988/1985] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:49:56 2022] DELETE /identity/v3/projects/5243c296c8cb4467bfd269e4622e1adb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:49:56.802439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:49:56.803840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:56.817229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feeda16-4cad-42d6-b4f1-f1f531b88de5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:49:56.863777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:49:56.874808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:49:56.875431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:56.875703 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:56.875968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] PATH_INFO: `/v3/users/264583661f414bdfafc3f0a14faccf44` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:56.877054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=264583661f414bdfafc3f0a14faccf44)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:49:56.879009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:49:56.926453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '264583661f414bdfafc3f0a14faccf44'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.927177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '264583661f414bdfafc3f0a14faccf44'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.939592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '264583661f414bdfafc3f0a14faccf44'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.940278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '264583661f414bdfafc3f0a14faccf44'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.947624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '264583661f414bdfafc3f0a14faccf44'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:56.967023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 998/1986] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:49:56 2022] DELETE /identity/v3/users/264583661f414bdfafc3f0a14faccf44 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:49:57.149715 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:49:57.151148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:49:57.178913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b505725-d537-4c29-b7e1-707a0e292045 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:49:57.237226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:49:57.237865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:49:57.238149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:49:57.238453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] PATH_INFO: `/v3/projects/76432deb97844d819f955521555aa08c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:49:57.240286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=76432deb97844d819f955521555aa08c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:49:57.242277 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:49:57.287902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76432deb97844d819f955521555aa08c', 'request_id': 'req-7e9b7527-51be-4a40-9fbc-8219b03207ca'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:57.294734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76432deb97844d819f955521555aa08c', 'request_id': 'req-7e9b7527-51be-4a40-9fbc-8219b03207ca'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:57.300796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76432deb97844d819f955521555aa08c', 'request_id': 'req-7e9b7527-51be-4a40-9fbc-8219b03207ca'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:57.311544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] The token cache is being invalidate because project 76432deb97844d819f955521555aa08c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:49:57.311830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:57.312461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:49:57.314029 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 989/1987] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:49:57 2022] DELETE /identity/v3/projects/76432deb97844d819f955521555aa08c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:00.561166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:00.563036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:00.634919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e1d09-cbd6-456a-9ce1-88c75212a201 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:00.700198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac37b6-5b4f-4603-bf83-493a0972d68d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:00.700819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac37b6-5b4f-4603-bf83-493a0972d68d service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:00.701099 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac37b6-5b4f-4603-bf83-493a0972d68d service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:00.701397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac37b6-5b4f-4603-bf83-493a0972d68d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:00.703165 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-87ac37b6-5b4f-4603-bf83-493a0972d68d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:00.715177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac37b6-5b4f-4603-bf83-493a0972d68d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:00.716750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac37b6-5b4f-4603-bf83-493a0972d68d service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:00.727455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 999/1988] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:50:07.308553 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=67,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:07.508683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:07.510104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:07.600418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b7527-51be-4a40-9fbc-8219b03207ca admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:07.604190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b3ea2-c89f-446b-afe8-f53ca1835642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:07.604799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b3ea2-c89f-446b-afe8-f53ca1835642 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:07.605076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b3ea2-c89f-446b-afe8-f53ca1835642 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:07.605346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b3ea2-c89f-446b-afe8-f53ca1835642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:07.607097 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3e1b3ea2-c89f-446b-afe8-f53ca1835642 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:07.672196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b3ea2-c89f-446b-afe8-f53ca1835642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:07.673622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b3ea2-c89f-446b-afe8-f53ca1835642 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:07.684154 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 990/1989] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:10.722522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:11.352813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac37b6-5b4f-4603-bf83-493a0972d68d service placement] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:11.354273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-87ac37b6-5b4f-4603-bf83-493a0972d68d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:11.369621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac37b6-5b4f-4603-bf83-493a0972d68d service placement] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:11.374679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41edbf-1181-486d-adbc-a50e9bc121be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:11.375357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41edbf-1181-486d-adbc-a50e9bc121be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:11.375634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41edbf-1181-486d-adbc-a50e9bc121be admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:11.375904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41edbf-1181-486d-adbc-a50e9bc121be admin admin] PATH_INFO: `/v3/users/be793f4685f94beba74adedbc02ee50a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:11.377032 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41edbf-1181-486d-adbc-a50e9bc121be admin admin] RBAC: Authorizing `identity:delete_user(user_id=be793f4685f94beba74adedbc02ee50a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:11.379808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41edbf-1181-486d-adbc-a50e9bc121be admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:11.433041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6a41edbf-1181-486d-adbc-a50e9bc121be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be793f4685f94beba74adedbc02ee50a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:11.433716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6a41edbf-1181-486d-adbc-a50e9bc121be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be793f4685f94beba74adedbc02ee50a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:11.446632 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6a41edbf-1181-486d-adbc-a50e9bc121be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be793f4685f94beba74adedbc02ee50a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:11.447276 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6a41edbf-1181-486d-adbc-a50e9bc121be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be793f4685f94beba74adedbc02ee50a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:11.454374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6a41edbf-1181-486d-adbc-a50e9bc121be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be793f4685f94beba74adedbc02ee50a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:11.466722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1000/1990] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:11 2022] DELETE /identity/v3/users/be793f4685f94beba74adedbc02ee50a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:11.640017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b3ea2-c89f-446b-afe8-f53ca1835642 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:11.650806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b3ea2-c89f-446b-afe8-f53ca1835642 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:11.707930 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:11.708557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:11.708836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:11.709103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] PATH_INFO: `/v3/projects/d506ce8ca200496cbc15d09f49eab942` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:11.710267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d506ce8ca200496cbc15d09f49eab942)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:11.712183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:11.747709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:11.761952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd506ce8ca200496cbc15d09f49eab942', 'request_id': 'req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:11.769197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd506ce8ca200496cbc15d09f49eab942', 'request_id': 'req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:11.775635 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd506ce8ca200496cbc15d09f49eab942', 'request_id': 'req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:11.786838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] The token cache is being invalidate because project d506ce8ca200496cbc15d09f49eab942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:11.787120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:11.787750 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3cd97a86-1038-4b3d-85f5-ba4b28df37ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:11.789222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 991/1991] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:11 2022] DELETE /identity/v3/projects/d506ce8ca200496cbc15d09f49eab942 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:11.794400 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f535f-fa92-4638-a0fc-c1469f2d3d1e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:11.794889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f535f-fa92-4638-a0fc-c1469f2d3d1e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:11.794997 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f535f-fa92-4638-a0fc-c1469f2d3d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:11.852415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e31f535f-fa92-4638-a0fc-c1469f2d3d1e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:11.853631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e31f535f-fa92-4638-a0fc-c1469f2d3d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:11.912390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1001/1992] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:11.920814 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0ed4a-69b8-43a8-bb0d-2142f576c063 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:11.921096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0ed4a-69b8-43a8-bb0d-2142f576c063 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:11.921365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0ed4a-69b8-43a8-bb0d-2142f576c063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:11.942803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-23e0ed4a-69b8-43a8-bb0d-2142f576c063 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:11.944029 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-23e0ed4a-69b8-43a8-bb0d-2142f576c063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:11.952062 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 992/1993] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:11.957079 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f535f-fa92-4638-a0fc-c1469f2d3d1e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:11.966699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f535f-fa92-4638-a0fc-c1469f2d3d1e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:11.971192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8b089-f983-44b4-ab92-7ed960f52638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:11.971783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8b089-f983-44b4-ab92-7ed960f52638 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:11.972058 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8b089-f983-44b4-ab92-7ed960f52638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:11.972326 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8b089-f983-44b4-ab92-7ed960f52638 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:11.973528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8b089-f983-44b4-ab92-7ed960f52638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:50:11.973827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8b089-f983-44b4-ab92-7ed960f52638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:11.975431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8b089-f983-44b4-ab92-7ed960f52638 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:11.986841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1002/1994] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:50:11.991754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0ed4a-69b8-43a8-bb0d-2142f576c063 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:12.000693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0ed4a-69b8-43a8-bb0d-2142f576c063 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:12.005461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41fd72-4cad-47f9-bd05-829229057f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:12.006061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41fd72-4cad-47f9-bd05-829229057f9b admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:12.006374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41fd72-4cad-47f9-bd05-829229057f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:12.006645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41fd72-4cad-47f9-bd05-829229057f9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:12.007284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41fd72-4cad-47f9-bd05-829229057f9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:12.009164 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41fd72-4cad-47f9-bd05-829229057f9b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:12.036310 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 993/1995] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:11 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:12.040469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8b089-f983-44b4-ab92-7ed960f52638 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:12.050240 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8b089-f983-44b4-ab92-7ed960f52638 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:12.078951 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:12.112670 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b83bb-f750-47c8-ab46-f9da773cf9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:12.113394 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b83bb-f750-47c8-ab46-f9da773cf9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:12.113688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b83bb-f750-47c8-ab46-f9da773cf9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:12.113961 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b83bb-f750-47c8-ab46-f9da773cf9d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:12.114631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b83bb-f750-47c8-ab46-f9da773cf9d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:12.116590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b83bb-f750-47c8-ab46-f9da773cf9d9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:12.137326 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1003/1996] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:12 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:12.141512 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41fd72-4cad-47f9-bd05-829229057f9b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:12.152550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41fd72-4cad-47f9-bd05-829229057f9b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:12.157368 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac136906-7aec-467e-bcc7-abe1f637ca93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:12.157958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac136906-7aec-467e-bcc7-abe1f637ca93 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:12.158271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac136906-7aec-467e-bcc7-abe1f637ca93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:12.158540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac136906-7aec-467e-bcc7-abe1f637ca93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:12.159174 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac136906-7aec-467e-bcc7-abe1f637ca93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:12.161370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac136906-7aec-467e-bcc7-abe1f637ca93 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:12.177439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 994/1997] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:12.181256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b83bb-f750-47c8-ab46-f9da773cf9d9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:12.192050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b83bb-f750-47c8-ab46-f9da773cf9d9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:12.196860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2de0e-2d22-489b-9d21-e1978ccb8a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:12.197485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2de0e-2d22-489b-9d21-e1978ccb8a3b admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:12.197764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2de0e-2d22-489b-9d21-e1978ccb8a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:12.198036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2de0e-2d22-489b-9d21-e1978ccb8a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:12.198776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2de0e-2d22-489b-9d21-e1978ccb8a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:12.200389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2de0e-2d22-489b-9d21-e1978ccb8a3b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:12.213986 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1004/1998] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:50:12.218274 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac136906-7aec-467e-bcc7-abe1f637ca93 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:12.236194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac136906-7aec-467e-bcc7-abe1f637ca93 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:12.241084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e22e35-f13f-4f4a-913d-d48b8e54df5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:12.241733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22e35-f13f-4f4a-913d-d48b8e54df5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:12.242012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22e35-f13f-4f4a-913d-d48b8e54df5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:12.242335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e22e35-f13f-4f4a-913d-d48b8e54df5d admin admin] PATH_INFO: `/v3/projects/b1f0143283124ac6848b0df6d4c2d225/users/ef2445015bec425db199fb986db76dee/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:12.258440 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e22e35-f13f-4f4a-913d-d48b8e54df5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f0143283124ac6848b0df6d4c2d225, user_id=ef2445015bec425db199fb986db76dee, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:12.261087 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e22e35-f13f-4f4a-913d-d48b8e54df5d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:12.279781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 995/1999] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:12 2022] PUT /identity/v3/projects/b1f0143283124ac6848b0df6d4c2d225/users/ef2445015bec425db199fb986db76dee/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:12.294540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963289c-c3ac-4d08-ba65-de1a5b306a1a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:12.294825 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963289c-c3ac-4d08-ba65-de1a5b306a1a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:12.295099 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963289c-c3ac-4d08-ba65-de1a5b306a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:12.325134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c963289c-c3ac-4d08-ba65-de1a5b306a1a None None] MFA Rules not processed for user `ef2445015bec425db199fb986db76dee`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:12.326660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c963289c-c3ac-4d08-ba65-de1a5b306a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:12.393802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1005/2000] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:12.409335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e22e35-f13f-4f4a-913d-d48b8e54df5d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:12.411606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-92e22e35-f13f-4f4a-913d-d48b8e54df5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:12.526320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e22e35-f13f-4f4a-913d-d48b8e54df5d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:12.532474 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cc99e-d41d-47bf-a01e-31cd139dc023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:12.533088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cc99e-d41d-47bf-a01e-31cd139dc023 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:12.533373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cc99e-d41d-47bf-a01e-31cd139dc023 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:12.534429 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cc99e-d41d-47bf-a01e-31cd139dc023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:12.542656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cc99e-d41d-47bf-a01e-31cd139dc023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:12.544243 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cc99e-d41d-47bf-a01e-31cd139dc023 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:12.554411 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 996/2001] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:15.271502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963289c-c3ac-4d08-ba65-de1a5b306a1a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:15.329140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963289c-c3ac-4d08-ba65-de1a5b306a1a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:15.334257 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:15.334872 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:15.335145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:15.335408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] PATH_INFO: `/v3/users/ef2445015bec425db199fb986db76dee` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:15.336500 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef2445015bec425db199fb986db76dee)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:15.338408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:15.377743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:15.385082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef2445015bec425db199fb986db76dee'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:15.385999 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef2445015bec425db199fb986db76dee'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:15.398374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef2445015bec425db199fb986db76dee'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:15.399002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef2445015bec425db199fb986db76dee'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:15.406080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef2445015bec425db199fb986db76dee'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:15.419401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1006/2002] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:15 2022] DELETE /identity/v3/users/ef2445015bec425db199fb986db76dee => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:15.423971 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604694c-d955-496d-b767-b6b5856c2583 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:15.424465 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604694c-d955-496d-b767-b6b5856c2583 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:15.424572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604694c-d955-496d-b767-b6b5856c2583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:15.443792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-d604694c-d955-496d-b767-b6b5856c2583 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:15.444976 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d604694c-d955-496d-b767-b6b5856c2583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:15.453916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:15.504265 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 997/2003] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:15.515009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:15.516387 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:15.573889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57233ee4-4087-4eb7-89ae-49f14f87fb57 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:15.577499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66303b3-bc14-4c65-a955-d16cb7e475d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:15.578090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66303b3-bc14-4c65-a955-d16cb7e475d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:15.578409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66303b3-bc14-4c65-a955-d16cb7e475d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:15.578662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66303b3-bc14-4c65-a955-d16cb7e475d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:15.588805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66303b3-bc14-4c65-a955-d16cb7e475d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:15.590258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66303b3-bc14-4c65-a955-d16cb7e475d5 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:15.600736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1007/2004] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:15.869749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604694c-d955-496d-b767-b6b5856c2583 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:15.871492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d604694c-d955-496d-b767-b6b5856c2583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:15.884152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604694c-d955-496d-b767-b6b5856c2583 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:15.888737 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:15.889369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:15.889646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:15.889912 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] PATH_INFO: `/v3/projects/b1f0143283124ac6848b0df6d4c2d225` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:15.891067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f0143283124ac6848b0df6d4c2d225)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:15.893014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:15.939204 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f0143283124ac6848b0df6d4c2d225', 'request_id': 'req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:15.947227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f0143283124ac6848b0df6d4c2d225', 'request_id': 'req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:15.953317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f0143283124ac6848b0df6d4c2d225', 'request_id': 'req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:15.963800 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] The token cache is being invalidate because project b1f0143283124ac6848b0df6d4c2d225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:15.964083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:15.964712 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8d5b0453-c3a3-4a85-9fc8-7c21d089f171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:15.966192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 998/2005] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:15 2022] DELETE /identity/v3/projects/b1f0143283124ac6848b0df6d4c2d225 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:15.971077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba9283-da7e-42a7-90c6-4e5dbe1b6027 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:15.971390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba9283-da7e-42a7-90c6-4e5dbe1b6027 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:15.971665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba9283-da7e-42a7-90c6-4e5dbe1b6027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:15.991209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-43ba9283-da7e-42a7-90c6-4e5dbe1b6027 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:15.992392 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-43ba9283-da7e-42a7-90c6-4e5dbe1b6027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:16.052703 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1008/2006] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:16.060664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c3965-8cc0-4350-afc4-ba61a33070ed None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.060948 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c3965-8cc0-4350-afc4-ba61a33070ed None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.061221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c3965-8cc0-4350-afc4-ba61a33070ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.082068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6c6c3965-8cc0-4350-afc4-ba61a33070ed None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:16.083321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6c6c3965-8cc0-4350-afc4-ba61a33070ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:16.091127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 999/2007] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:16.095325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba9283-da7e-42a7-90c6-4e5dbe1b6027 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.105477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba9283-da7e-42a7-90c6-4e5dbe1b6027 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.110060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e25b6-0a46-4b56-b88d-4b7298cb6bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.110682 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e25b6-0a46-4b56-b88d-4b7298cb6bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.110963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e25b6-0a46-4b56-b88d-4b7298cb6bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.111236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e25b6-0a46-4b56-b88d-4b7298cb6bd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.111878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e25b6-0a46-4b56-b88d-4b7298cb6bd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:50:16.112179 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e25b6-0a46-4b56-b88d-4b7298cb6bd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.114307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e25b6-0a46-4b56-b88d-4b7298cb6bd2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.125889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1009/2008] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:50:16.130705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c3965-8cc0-4350-afc4-ba61a33070ed None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.139695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c3965-8cc0-4350-afc4-ba61a33070ed None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.144596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c6d28-7355-4598-8523-18432ca420e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.145253 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c6d28-7355-4598-8523-18432ca420e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.145527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c6d28-7355-4598-8523-18432ca420e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.145824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c6d28-7355-4598-8523-18432ca420e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.146498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c6d28-7355-4598-8523-18432ca420e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.148393 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c6d28-7355-4598-8523-18432ca420e7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.178927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1000/2009] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:16 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:16.180773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e25b6-0a46-4b56-b88d-4b7298cb6bd2 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.190763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e25b6-0a46-4b56-b88d-4b7298cb6bd2 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.235650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:16.250627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acc501-7c35-4372-8e26-e5a7ef4f3edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.251242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acc501-7c35-4372-8e26-e5a7ef4f3edd admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.251522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acc501-7c35-4372-8e26-e5a7ef4f3edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.251794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acc501-7c35-4372-8e26-e5a7ef4f3edd admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.252440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acc501-7c35-4372-8e26-e5a7ef4f3edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.254374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acc501-7c35-4372-8e26-e5a7ef4f3edd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.282323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1010/2010] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:16 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:50:16.283524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c6d28-7355-4598-8523-18432ca420e7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.294283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c6d28-7355-4598-8523-18432ca420e7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.299648 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a0110-72fa-456a-9bd7-fd9947f2e517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.300245 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a0110-72fa-456a-9bd7-fd9947f2e517 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.300527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a0110-72fa-456a-9bd7-fd9947f2e517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.300796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a0110-72fa-456a-9bd7-fd9947f2e517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.302077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a0110-72fa-456a-9bd7-fd9947f2e517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.303748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a0110-72fa-456a-9bd7-fd9947f2e517 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.313498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1001/2011] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:50:16.317591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acc501-7c35-4372-8e26-e5a7ef4f3edd admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.327460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acc501-7c35-4372-8e26-e5a7ef4f3edd admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.332044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d2be5-0e69-4552-9bb3-137ca3c5f201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.332732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d2be5-0e69-4552-9bb3-137ca3c5f201 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.333314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d2be5-0e69-4552-9bb3-137ca3c5f201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.333422 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d2be5-0e69-4552-9bb3-137ca3c5f201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.334039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d2be5-0e69-4552-9bb3-137ca3c5f201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.335672 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d2be5-0e69-4552-9bb3-137ca3c5f201 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.345991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1011/2012] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:16.350492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a0110-72fa-456a-9bd7-fd9947f2e517 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.360311 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a0110-72fa-456a-9bd7-fd9947f2e517 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.365876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886017a-b206-4b1e-93ac-8462cca1c4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.366548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886017a-b206-4b1e-93ac-8462cca1c4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.366823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886017a-b206-4b1e-93ac-8462cca1c4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.367087 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886017a-b206-4b1e-93ac-8462cca1c4e5 admin admin] PATH_INFO: `/v3/projects/11fe23586978424e83b99ce62d4292fc/users/eb5d8298036d4ae1882c54c62b352696/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.381658 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886017a-b206-4b1e-93ac-8462cca1c4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fe23586978424e83b99ce62d4292fc, user_id=eb5d8298036d4ae1882c54c62b352696, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.384332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886017a-b206-4b1e-93ac-8462cca1c4e5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.397704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1002/2013] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:16 2022] PUT /identity/v3/projects/11fe23586978424e83b99ce62d4292fc/users/eb5d8298036d4ae1882c54c62b352696/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:50:16.409357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a920f6-d647-48ed-97ad-13672ee508e4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.409639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a920f6-d647-48ed-97ad-13672ee508e4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.409908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a920f6-d647-48ed-97ad-13672ee508e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.438942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e1a920f6-d647-48ed-97ad-13672ee508e4 None None] MFA Rules not processed for user `eb5d8298036d4ae1882c54c62b352696`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:16.440183 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e1a920f6-d647-48ed-97ad-13672ee508e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:16.503899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1012/2014] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:16.508507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886017a-b206-4b1e-93ac-8462cca1c4e5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.548138 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:16.572731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886017a-b206-4b1e-93ac-8462cca1c4e5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.577209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba9ab4-9ca6-4143-ab66-8bc9a732a6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.577814 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba9ab4-9ca6-4143-ab66-8bc9a732a6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.578093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba9ab4-9ca6-4143-ab66-8bc9a732a6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.578415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba9ab4-9ca6-4143-ab66-8bc9a732a6b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.579036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba9ab4-9ca6-4143-ab66-8bc9a732a6b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.580918 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba9ab4-9ca6-4143-ab66-8bc9a732a6b2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.607924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1003/2015] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:16 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:16.611865 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a920f6-d647-48ed-97ad-13672ee508e4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.622858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a920f6-d647-48ed-97ad-13672ee508e4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.679993 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bbf89-115b-411f-b788-38cf8b2020aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.680605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bbf89-115b-411f-b788-38cf8b2020aa admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.680883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bbf89-115b-411f-b788-38cf8b2020aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.681155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bbf89-115b-411f-b788-38cf8b2020aa admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.681789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bbf89-115b-411f-b788-38cf8b2020aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.683858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bbf89-115b-411f-b788-38cf8b2020aa admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.704169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1013/2016] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:16 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:16.708693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba9ab4-9ca6-4143-ab66-8bc9a732a6b2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.718867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba9ab4-9ca6-4143-ab66-8bc9a732a6b2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.724357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879faa98-93fd-401e-8817-5ca79eb4fad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.724961 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879faa98-93fd-401e-8817-5ca79eb4fad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.725241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879faa98-93fd-401e-8817-5ca79eb4fad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.725508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879faa98-93fd-401e-8817-5ca79eb4fad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.726145 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879faa98-93fd-401e-8817-5ca79eb4fad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.727757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879faa98-93fd-401e-8817-5ca79eb4fad7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.738444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1004/2017] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:16.742593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bbf89-115b-411f-b788-38cf8b2020aa admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.752443 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bbf89-115b-411f-b788-38cf8b2020aa admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.757051 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7de14-a162-490e-97eb-bc789cd23c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.757696 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7de14-a162-490e-97eb-bc789cd23c83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.757975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7de14-a162-490e-97eb-bc789cd23c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.758281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7de14-a162-490e-97eb-bc789cd23c83 admin admin] PATH_INFO: `/v3/projects/d1147802da284491b4dee1bbf9f61fe1/users/19983229a2dd4738aa84d932e003d660/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.773224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7de14-a162-490e-97eb-bc789cd23c83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1147802da284491b4dee1bbf9f61fe1, user_id=19983229a2dd4738aa84d932e003d660, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.775894 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7de14-a162-490e-97eb-bc789cd23c83 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.789870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1014/2018] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:16 2022] PUT /identity/v3/projects/d1147802da284491b4dee1bbf9f61fe1/users/19983229a2dd4738aa84d932e003d660/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:16.793774 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879faa98-93fd-401e-8817-5ca79eb4fad7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.804135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879faa98-93fd-401e-8817-5ca79eb4fad7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.860291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e8555-3dbf-4aad-82f9-0501b22db5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.860814 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e8555-3dbf-4aad-82f9-0501b22db5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.861087 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e8555-3dbf-4aad-82f9-0501b22db5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.861353 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e8555-3dbf-4aad-82f9-0501b22db5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.861977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e8555-3dbf-4aad-82f9-0501b22db5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.863580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e8555-3dbf-4aad-82f9-0501b22db5e6 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.872879 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1005/2019] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:16.877116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7de14-a162-490e-97eb-bc789cd23c83 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:16.888474 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7de14-a162-490e-97eb-bc789cd23c83 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:16.893088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d531a6-6585-4a88-89d7-7b745a0918f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:16.893732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d531a6-6585-4a88-89d7-7b745a0918f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.894009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d531a6-6585-4a88-89d7-7b745a0918f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.894306 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d531a6-6585-4a88-89d7-7b745a0918f4 admin admin] PATH_INFO: `/v3/projects/d1147802da284491b4dee1bbf9f61fe1/users/19983229a2dd4738aa84d932e003d660/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.897012 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d531a6-6585-4a88-89d7-7b745a0918f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1147802da284491b4dee1bbf9f61fe1, user_id=19983229a2dd4738aa84d932e003d660, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:16.899670 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d531a6-6585-4a88-89d7-7b745a0918f4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:16.910108 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1015/2020] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:16 2022] PUT /identity/v3/projects/d1147802da284491b4dee1bbf9f61fe1/users/19983229a2dd4738aa84d932e003d660/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:50:16.920959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8a13d-0aaf-4e3b-8560-c82c4105687e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:16.921240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8a13d-0aaf-4e3b-8560-c82c4105687e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:16.921513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8a13d-0aaf-4e3b-8560-c82c4105687e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:16.949353 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-cfe8a13d-0aaf-4e3b-8560-c82c4105687e None None] MFA Rules not processed for user `19983229a2dd4738aa84d932e003d660`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:16.950565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cfe8a13d-0aaf-4e3b-8560-c82c4105687e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:17.018966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1006/2021] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:17.030156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d531a6-6585-4a88-89d7-7b745a0918f4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:17.032100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-67d531a6-6585-4a88-89d7-7b745a0918f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:17.042373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:17.111917 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d531a6-6585-4a88-89d7-7b745a0918f4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:17.115869 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d764955-d9ee-408f-b2d9-443f62e20b48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:17.116463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d764955-d9ee-408f-b2d9-443f62e20b48 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:17.116738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d764955-d9ee-408f-b2d9-443f62e20b48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:17.117003 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d764955-d9ee-408f-b2d9-443f62e20b48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:17.125649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d764955-d9ee-408f-b2d9-443f62e20b48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:17.127069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d764955-d9ee-408f-b2d9-443f62e20b48 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:17.137120 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1016/2022] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:17.272721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8a13d-0aaf-4e3b-8560-c82c4105687e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:17.305610 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:17.335466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8a13d-0aaf-4e3b-8560-c82c4105687e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:17.339902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:17.340520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:17.340793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:17.341057 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] PATH_INFO: `/v3/users/eb5d8298036d4ae1882c54c62b352696` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:17.342149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb5d8298036d4ae1882c54c62b352696)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:17.344063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:17.390188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb5d8298036d4ae1882c54c62b352696'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:17.390764 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb5d8298036d4ae1882c54c62b352696'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:17.391356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb5d8298036d4ae1882c54c62b352696'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:17.398704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb5d8298036d4ae1882c54c62b352696'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:17.410579 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb5d8298036d4ae1882c54c62b352696'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:17.422847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1007/2023] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:17 2022] DELETE /identity/v3/users/eb5d8298036d4ae1882c54c62b352696 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:17.428145 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469d95d-d670-424a-b6c0-e52b722b88ed None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:17.428625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469d95d-d670-424a-b6c0-e52b722b88ed None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:17.428730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469d95d-d670-424a-b6c0-e52b722b88ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:17.447972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3469d95d-d670-424a-b6c0-e52b722b88ed None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:17.449163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3469d95d-d670-424a-b6c0-e52b722b88ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:17.507093 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1017/2024] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:17.516819 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:17.518253 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:17.575921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8b997-7668-4093-86c9-fb5738038b8a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:17.579549 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a348534-91e3-4627-98d9-1ddbb66a15c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:17.580151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a348534-91e3-4627-98d9-1ddbb66a15c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:17.580427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a348534-91e3-4627-98d9-1ddbb66a15c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:17.580695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a348534-91e3-4627-98d9-1ddbb66a15c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:17.589855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a348534-91e3-4627-98d9-1ddbb66a15c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:17.591318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a348534-91e3-4627-98d9-1ddbb66a15c6 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:17.601048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1008/2025] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:18.124322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469d95d-d670-424a-b6c0-e52b722b88ed None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:18.125716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3469d95d-d670-424a-b6c0-e52b722b88ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:18.137679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469d95d-d670-424a-b6c0-e52b722b88ed None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:18.142257 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:18.142894 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:18.143170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:18.143434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] PATH_INFO: `/v3/projects/11fe23586978424e83b99ce62d4292fc` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:18.144533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fe23586978424e83b99ce62d4292fc)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:18.147343 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:18.194749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fe23586978424e83b99ce62d4292fc', 'request_id': 'req-11e699d1-5a5b-4363-9557-918d8d134ffc'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.200887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fe23586978424e83b99ce62d4292fc', 'request_id': 'req-11e699d1-5a5b-4363-9557-918d8d134ffc'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.209425 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fe23586978424e83b99ce62d4292fc', 'request_id': 'req-11e699d1-5a5b-4363-9557-918d8d134ffc'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.220862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] The token cache is being invalidate because project 11fe23586978424e83b99ce62d4292fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:18.221150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.221777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.224085 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1018/2026] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:18 2022] DELETE /identity/v3/projects/11fe23586978424e83b99ce62d4292fc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:18.228276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a348534-91e3-4627-98d9-1ddbb66a15c6 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:18.237896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a348534-91e3-4627-98d9-1ddbb66a15c6 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:18.284687 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:18.295162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dcf15-f97b-4b31-8ea9-541dfe663e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:18.295785 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dcf15-f97b-4b31-8ea9-541dfe663e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:18.296058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dcf15-f97b-4b31-8ea9-541dfe663e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:18.296326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dcf15-f97b-4b31-8ea9-541dfe663e0c admin admin] PATH_INFO: `/v3/users/19983229a2dd4738aa84d932e003d660` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:18.297452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532dcf15-f97b-4b31-8ea9-541dfe663e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=19983229a2dd4738aa84d932e003d660)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:18.299377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532dcf15-f97b-4b31-8ea9-541dfe663e0c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:18.344668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-532dcf15-f97b-4b31-8ea9-541dfe663e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19983229a2dd4738aa84d932e003d660'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.345309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-532dcf15-f97b-4b31-8ea9-541dfe663e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19983229a2dd4738aa84d932e003d660'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.345905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-532dcf15-f97b-4b31-8ea9-541dfe663e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19983229a2dd4738aa84d932e003d660'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.353695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-532dcf15-f97b-4b31-8ea9-541dfe663e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19983229a2dd4738aa84d932e003d660'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.365502 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-532dcf15-f97b-4b31-8ea9-541dfe663e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19983229a2dd4738aa84d932e003d660'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.378235 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1009/2027] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:18 2022] DELETE /identity/v3/users/19983229a2dd4738aa84d932e003d660 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:18.653648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:18.655869 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:18.671448 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e699d1-5a5b-4363-9557-918d8d134ffc admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:18.680711 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:18.731601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:18.732163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:18.732440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:18.732708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] PATH_INFO: `/v3/projects/d1147802da284491b4dee1bbf9f61fe1` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:18.733817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1147802da284491b4dee1bbf9f61fe1)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:18.735741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:18.782334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1147802da284491b4dee1bbf9f61fe1', 'request_id': 'req-ea8775af-452d-42e8-8177-d133025a4238'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.787962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1147802da284491b4dee1bbf9f61fe1', 'request_id': 'req-ea8775af-452d-42e8-8177-d133025a4238'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.794831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1147802da284491b4dee1bbf9f61fe1', 'request_id': 'req-ea8775af-452d-42e8-8177-d133025a4238'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.806832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] The token cache is being invalidate because project d1147802da284491b4dee1bbf9f61fe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:18.807120 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.807744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-ea8775af-452d-42e8-8177-d133025a4238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:18.809271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1019/2028] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:18 2022] DELETE /identity/v3/projects/d1147802da284491b4dee1bbf9f61fe1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:18.815113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfffac9-8593-45fc-9624-a343a8c692ae None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:18.815573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfffac9-8593-45fc-9624-a343a8c692ae None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:18.815686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfffac9-8593-45fc-9624-a343a8c692ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:18.835378 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4dfffac9-8593-45fc-9624-a343a8c692ae None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:18.836560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4dfffac9-8593-45fc-9624-a343a8c692ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:18.893851 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1010/2029] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:18.901839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6570a2-f2b9-4814-a950-d3cfc4dd34db None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:18.902127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6570a2-f2b9-4814-a950-d3cfc4dd34db None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:18.902446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6570a2-f2b9-4814-a950-d3cfc4dd34db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:18.922364 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7c6570a2-f2b9-4814-a950-d3cfc4dd34db None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:18.923551 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7c6570a2-f2b9-4814-a950-d3cfc4dd34db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:18.932054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1020/2030] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:50:18.936315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfffac9-8593-45fc-9624-a343a8c692ae None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:18.946690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfffac9-8593-45fc-9624-a343a8c692ae None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:18.951210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6254c49-c515-46f2-b0d5-eefe978afa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:18.951806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6254c49-c515-46f2-b0d5-eefe978afa47 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:18.952080 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6254c49-c515-46f2-b0d5-eefe978afa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:18.952345 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6254c49-c515-46f2-b0d5-eefe978afa47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:18.952982 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6254c49-c515-46f2-b0d5-eefe978afa47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:50:18.953280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6254c49-c515-46f2-b0d5-eefe978afa47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:18.955100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6254c49-c515-46f2-b0d5-eefe978afa47 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:18.967473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1011/2031] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:50:18.971624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6570a2-f2b9-4814-a950-d3cfc4dd34db None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:18.981113 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6570a2-f2b9-4814-a950-d3cfc4dd34db None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:18.985763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9f0c4-10e1-4f66-86bb-32412e05690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:18.986325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9f0c4-10e1-4f66-86bb-32412e05690a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:18.986599 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9f0c4-10e1-4f66-86bb-32412e05690a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:18.986865 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9f0c4-10e1-4f66-86bb-32412e05690a admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:18.987494 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9f0c4-10e1-4f66-86bb-32412e05690a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:18.990300 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9f0c4-10e1-4f66-86bb-32412e05690a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:19.017824 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1021/2032] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:18 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:19.022035 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6254c49-c515-46f2-b0d5-eefe978afa47 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:19.032051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6254c49-c515-46f2-b0d5-eefe978afa47 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:19.088562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bbd77-756c-4d2b-bd25-eb2272a2e599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:19.089171 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbd77-756c-4d2b-bd25-eb2272a2e599 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.089446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbd77-756c-4d2b-bd25-eb2272a2e599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.089714 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbd77-756c-4d2b-bd25-eb2272a2e599 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.090376 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bbd77-756c-4d2b-bd25-eb2272a2e599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:19.092270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bbd77-756c-4d2b-bd25-eb2272a2e599 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:19.100556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:19.123092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1012/2033] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:19 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:19.127078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9f0c4-10e1-4f66-86bb-32412e05690a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:19.146890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9f0c4-10e1-4f66-86bb-32412e05690a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:19.151793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e1e75-e4ca-4a33-a796-57a6eca80923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:19.152401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e1e75-e4ca-4a33-a796-57a6eca80923 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.152733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e1e75-e4ca-4a33-a796-57a6eca80923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.153002 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e1e75-e4ca-4a33-a796-57a6eca80923 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.153643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e1e75-e4ca-4a33-a796-57a6eca80923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:19.155366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e1e75-e4ca-4a33-a796-57a6eca80923 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:19.166945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1022/2034] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:19.170962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bbd77-756c-4d2b-bd25-eb2272a2e599 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:19.181523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bbd77-756c-4d2b-bd25-eb2272a2e599 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:19.186370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5a0bd-19f5-4321-855c-22dfe75ac8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:19.186848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5a0bd-19f5-4321-855c-22dfe75ac8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.187120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5a0bd-19f5-4321-855c-22dfe75ac8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.187390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5a0bd-19f5-4321-855c-22dfe75ac8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.188024 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5a0bd-19f5-4321-855c-22dfe75ac8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:19.189588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5a0bd-19f5-4321-855c-22dfe75ac8d3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:19.201650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1013/2035] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:19.206032 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e1e75-e4ca-4a33-a796-57a6eca80923 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:19.219973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e1e75-e4ca-4a33-a796-57a6eca80923 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:19.225468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40ecf6-090a-42a7-bebf-ffc12624053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:19.226126 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40ecf6-090a-42a7-bebf-ffc12624053c admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.226436 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40ecf6-090a-42a7-bebf-ffc12624053c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.226747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40ecf6-090a-42a7-bebf-ffc12624053c admin admin] PATH_INFO: `/v3/projects/3314054fe6ef471f8ae418f686dedba0/users/66ec37e738894469b2b2bfd0a8fdefa6/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.242245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40ecf6-090a-42a7-bebf-ffc12624053c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3314054fe6ef471f8ae418f686dedba0, user_id=66ec37e738894469b2b2bfd0a8fdefa6, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:19.244828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40ecf6-090a-42a7-bebf-ffc12624053c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:19.258146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1023/2036] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:19 2022] PUT /identity/v3/projects/3314054fe6ef471f8ae418f686dedba0/users/66ec37e738894469b2b2bfd0a8fdefa6/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:19.269570 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d612edd-20d9-455b-8db9-2dcbf703f00b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.270100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d612edd-20d9-455b-8db9-2dcbf703f00b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.270560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d612edd-20d9-455b-8db9-2dcbf703f00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.303272 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1d612edd-20d9-455b-8db9-2dcbf703f00b None None] MFA Rules not processed for user `66ec37e738894469b2b2bfd0a8fdefa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:19.304438 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1d612edd-20d9-455b-8db9-2dcbf703f00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:19.375119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1014/2037] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:19.380055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40ecf6-090a-42a7-bebf-ffc12624053c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:19.432411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:19.474426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40ecf6-090a-42a7-bebf-ffc12624053c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:19.474426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193cb69-a384-4de2-97fe-fadbe02b61da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:19.474426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193cb69-a384-4de2-97fe-fadbe02b61da admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.474426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193cb69-a384-4de2-97fe-fadbe02b61da admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.475070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193cb69-a384-4de2-97fe-fadbe02b61da admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.475070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193cb69-a384-4de2-97fe-fadbe02b61da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:19.475070 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193cb69-a384-4de2-97fe-fadbe02b61da admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:19.501614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1024/2038] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:19 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:19.505913 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d612edd-20d9-455b-8db9-2dcbf703f00b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:19.516272 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d612edd-20d9-455b-8db9-2dcbf703f00b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:19.575354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7a5a9-7fc4-4b07-8ab7-9fa6595385a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:19.575964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7a5a9-7fc4-4b07-8ab7-9fa6595385a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.576343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7a5a9-7fc4-4b07-8ab7-9fa6595385a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.576497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7a5a9-7fc4-4b07-8ab7-9fa6595385a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.577147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7a5a9-7fc4-4b07-8ab7-9fa6595385a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:19.579137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7a5a9-7fc4-4b07-8ab7-9fa6595385a2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:19.601406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1015/2039] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:19 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:50:19.605327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193cb69-a384-4de2-97fe-fadbe02b61da admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:19.615492 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193cb69-a384-4de2-97fe-fadbe02b61da admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:19.620122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f45f99-3018-4d23-8e93-93af9137435c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:19.620708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f45f99-3018-4d23-8e93-93af9137435c admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.620985 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f45f99-3018-4d23-8e93-93af9137435c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.621257 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f45f99-3018-4d23-8e93-93af9137435c admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.621888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f45f99-3018-4d23-8e93-93af9137435c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:19.623504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f45f99-3018-4d23-8e93-93af9137435c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:19.633940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1025/2040] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:19.637883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7a5a9-7fc4-4b07-8ab7-9fa6595385a2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:19.648365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7a5a9-7fc4-4b07-8ab7-9fa6595385a2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:19.653160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4039ed6-90a8-4861-b965-733f608e7cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:19.653838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4039ed6-90a8-4861-b965-733f608e7cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.654116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4039ed6-90a8-4861-b965-733f608e7cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.654543 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4039ed6-90a8-4861-b965-733f608e7cbc admin admin] PATH_INFO: `/v3/projects/c0be0368f5c14961a441d8ebcbde87c0/users/10bb26143ce14401916e868deab68e38/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.670411 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4039ed6-90a8-4861-b965-733f608e7cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0be0368f5c14961a441d8ebcbde87c0, user_id=10bb26143ce14401916e868deab68e38, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:19.673096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4039ed6-90a8-4861-b965-733f608e7cbc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:19.686640 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1016/2041] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:19 2022] PUT /identity/v3/projects/c0be0368f5c14961a441d8ebcbde87c0/users/10bb26143ce14401916e868deab68e38/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:19.690612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f45f99-3018-4d23-8e93-93af9137435c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:19.700897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f45f99-3018-4d23-8e93-93af9137435c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:19.761327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab26b5b-b2a5-4b8e-a683-f87b1abbe936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:19.761854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab26b5b-b2a5-4b8e-a683-f87b1abbe936 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.762130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab26b5b-b2a5-4b8e-a683-f87b1abbe936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.762434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab26b5b-b2a5-4b8e-a683-f87b1abbe936 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.763077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab26b5b-b2a5-4b8e-a683-f87b1abbe936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:19.764662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab26b5b-b2a5-4b8e-a683-f87b1abbe936 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:19.775564 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1026/2042] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:19.779850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4039ed6-90a8-4861-b965-733f608e7cbc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:19.791356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4039ed6-90a8-4861-b965-733f608e7cbc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:19.796027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea67ea7-f831-4b2a-b704-3082b698e249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:19.796726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea67ea7-f831-4b2a-b704-3082b698e249 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.796952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea67ea7-f831-4b2a-b704-3082b698e249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.797219 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea67ea7-f831-4b2a-b704-3082b698e249 admin admin] PATH_INFO: `/v3/projects/c0be0368f5c14961a441d8ebcbde87c0/users/10bb26143ce14401916e868deab68e38/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.799258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea67ea7-f831-4b2a-b704-3082b698e249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0be0368f5c14961a441d8ebcbde87c0, user_id=10bb26143ce14401916e868deab68e38, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:19.803075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea67ea7-f831-4b2a-b704-3082b698e249 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:19.813805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1017/2043] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:19 2022] PUT /identity/v3/projects/c0be0368f5c14961a441d8ebcbde87c0/users/10bb26143ce14401916e868deab68e38/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:50:19.825109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af43623-477b-463c-938d-b8b7a2560f3f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:19.825395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af43623-477b-463c-938d-b8b7a2560f3f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:19.825667 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af43623-477b-463c-938d-b8b7a2560f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:19.853840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9af43623-477b-463c-938d-b8b7a2560f3f None None] MFA Rules not processed for user `10bb26143ce14401916e868deab68e38`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:19.855055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9af43623-477b-463c-938d-b8b7a2560f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:19.921110 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1027/2044] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:19.932605 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea67ea7-f831-4b2a-b704-3082b698e249 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:19.934446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-aea67ea7-f831-4b2a-b704-3082b698e249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:19.958178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:19.996147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea67ea7-f831-4b2a-b704-3082b698e249 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:19.999799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401d6b6-f30e-4101-83ea-9e05f8c1e8ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:20.000402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401d6b6-f30e-4101-83ea-9e05f8c1e8ab service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:20.000680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401d6b6-f30e-4101-83ea-9e05f8c1e8ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:20.000949 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401d6b6-f30e-4101-83ea-9e05f8c1e8ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:20.056969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401d6b6-f30e-4101-83ea-9e05f8c1e8ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:20.058420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401d6b6-f30e-4101-83ea-9e05f8c1e8ab service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:20.067130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1018/2045] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:22.759176 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af43623-477b-463c-938d-b8b7a2560f3f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:22.797003 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:22.822465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af43623-477b-463c-938d-b8b7a2560f3f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:22.826853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:22.827472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:22.827744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:22.828012 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] PATH_INFO: `/v3/users/66ec37e738894469b2b2bfd0a8fdefa6` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:22.829102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ec37e738894469b2b2bfd0a8fdefa6)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:22.831041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:22.876591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ec37e738894469b2b2bfd0a8fdefa6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:22.877155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ec37e738894469b2b2bfd0a8fdefa6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:22.889237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ec37e738894469b2b2bfd0a8fdefa6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:22.889855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ec37e738894469b2b2bfd0a8fdefa6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:22.897120 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ec37e738894469b2b2bfd0a8fdefa6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:22.910896 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1028/2046] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:22 2022] DELETE /identity/v3/users/66ec37e738894469b2b2bfd0a8fdefa6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:22.915632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c5d3-4188-45d0-960f-655f40e6cc0c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:22.916131 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c5d3-4188-45d0-960f-655f40e6cc0c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:22.916230 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6c5d3-4188-45d0-960f-655f40e6cc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:22.935215 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-34a6c5d3-4188-45d0-960f-655f40e6cc0c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:22.936437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-34a6c5d3-4188-45d0-960f-655f40e6cc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:22.993653 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1019/2047] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:23.003335 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:23.004708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:23.060842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300e355-1a66-4faf-bc98-9d958b2b2c29 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:23.064457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c0779-f979-4319-9702-a9b52b06ec74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:23.065056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c0779-f979-4319-9702-a9b52b06ec74 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:23.065328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c0779-f979-4319-9702-a9b52b06ec74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:23.065595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c0779-f979-4319-9702-a9b52b06ec74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:23.075444 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c0779-f979-4319-9702-a9b52b06ec74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:23.076844 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c0779-f979-4319-9702-a9b52b06ec74 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:23.086780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1029/2048] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:23.349459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6c5d3-4188-45d0-960f-655f40e6cc0c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:23.351064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-34a6c5d3-4188-45d0-960f-655f40e6cc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:23.364197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6c5d3-4188-45d0-960f-655f40e6cc0c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:23.370030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:23.370708 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:23.370987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:23.371260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] PATH_INFO: `/v3/projects/3314054fe6ef471f8ae418f686dedba0` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:23.372395 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3314054fe6ef471f8ae418f686dedba0)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:23.374336 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:23.423118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3314054fe6ef471f8ae418f686dedba0', 'request_id': 'req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:23.431237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3314054fe6ef471f8ae418f686dedba0', 'request_id': 'req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:23.437609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3314054fe6ef471f8ae418f686dedba0', 'request_id': 'req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:23.444642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:23.450734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] The token cache is being invalidate because project 3314054fe6ef471f8ae418f686dedba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:23.451052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:23.451859 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:23.453970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1020/2049] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:23 2022] DELETE /identity/v3/projects/3314054fe6ef471f8ae418f686dedba0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:23.457955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c0779-f979-4319-9702-a9b52b06ec74 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:23.467503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c0779-f979-4319-9702-a9b52b06ec74 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:23.517330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:23.524885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90611b-cea4-4bd6-ad1a-ad18e35fde88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:23.525868 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90611b-cea4-4bd6-ad1a-ad18e35fde88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:23.526395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90611b-cea4-4bd6-ad1a-ad18e35fde88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:23.526841 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90611b-cea4-4bd6-ad1a-ad18e35fde88 admin admin] PATH_INFO: `/v3/users/10bb26143ce14401916e868deab68e38` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:23.528199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90611b-cea4-4bd6-ad1a-ad18e35fde88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10bb26143ce14401916e868deab68e38)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:23.530126 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90611b-cea4-4bd6-ad1a-ad18e35fde88 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:23.576278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0e90611b-cea4-4bd6-ad1a-ad18e35fde88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10bb26143ce14401916e868deab68e38'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:23.576910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0e90611b-cea4-4bd6-ad1a-ad18e35fde88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10bb26143ce14401916e868deab68e38'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:23.589065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0e90611b-cea4-4bd6-ad1a-ad18e35fde88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10bb26143ce14401916e868deab68e38'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:23.589686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0e90611b-cea4-4bd6-ad1a-ad18e35fde88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10bb26143ce14401916e868deab68e38'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:23.596571 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0e90611b-cea4-4bd6-ad1a-ad18e35fde88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10bb26143ce14401916e868deab68e38'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:23.609537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1030/2050] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:23 2022] DELETE /identity/v3/users/10bb26143ce14401916e868deab68e38 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:23.886882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:23.888439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:23.901520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a28c4-1f83-4c2b-9f25-b1d2c3c348d2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:23.959149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:23.959767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:23.959884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:23.960115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] PATH_INFO: `/v3/projects/c0be0368f5c14961a441d8ebcbde87c0` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:23.961234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0be0368f5c14961a441d8ebcbde87c0)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:23.963166 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.009729 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0be0368f5c14961a441d8ebcbde87c0', 'request_id': 'req-742e491a-23ae-4bb4-b552-a58f5f0de898'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:24.016322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0be0368f5c14961a441d8ebcbde87c0', 'request_id': 'req-742e491a-23ae-4bb4-b552-a58f5f0de898'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:24.023118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0be0368f5c14961a441d8ebcbde87c0', 'request_id': 'req-742e491a-23ae-4bb4-b552-a58f5f0de898'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:24.039108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] The token cache is being invalidate because project c0be0368f5c14961a441d8ebcbde87c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:24.039396 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:24.040053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-742e491a-23ae-4bb4-b552-a58f5f0de898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:24.041536 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1021/2051] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:23 2022] DELETE /identity/v3/projects/c0be0368f5c14961a441d8ebcbde87c0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:24.046273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5ef74-7299-493d-a67c-1985f6445496 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.046746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5ef74-7299-493d-a67c-1985f6445496 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.046842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5ef74-7299-493d-a67c-1985f6445496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.066046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3dc5ef74-7299-493d-a67c-1985f6445496 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:24.067256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3dc5ef74-7299-493d-a67c-1985f6445496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:24.125469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1031/2052] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:24.133052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807fe19-200d-4382-9caa-827173ef085a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.133341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807fe19-200d-4382-9caa-827173ef085a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.133619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807fe19-200d-4382-9caa-827173ef085a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.154926 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e807fe19-200d-4382-9caa-827173ef085a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:24.156389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e807fe19-200d-4382-9caa-827173ef085a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:24.164981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1022/2053] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:24.169174 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5ef74-7299-493d-a67c-1985f6445496 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:24.179273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5ef74-7299-493d-a67c-1985f6445496 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:24.183839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4fc8d-ac78-488e-b000-ab63a2088d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:24.184460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4fc8d-ac78-488e-b000-ab63a2088d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.184737 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4fc8d-ac78-488e-b000-ab63a2088d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.185004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4fc8d-ac78-488e-b000-ab63a2088d1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.185672 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb4fc8d-ac78-488e-b000-ab63a2088d1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:50:24.185974 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb4fc8d-ac78-488e-b000-ab63a2088d1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:24.187670 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb4fc8d-ac78-488e-b000-ab63a2088d1e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.201391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1032/2054] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:50:24.205691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807fe19-200d-4382-9caa-827173ef085a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:24.215920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807fe19-200d-4382-9caa-827173ef085a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:24.220513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21556cd4-acdc-41cb-9857-dd713bba11f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:24.221108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21556cd4-acdc-41cb-9857-dd713bba11f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.221382 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21556cd4-acdc-41cb-9857-dd713bba11f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.221645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21556cd4-acdc-41cb-9857-dd713bba11f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.222307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21556cd4-acdc-41cb-9857-dd713bba11f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:24.224184 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21556cd4-acdc-41cb-9857-dd713bba11f7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.250906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1023/2055] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:24 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:24.255030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4fc8d-ac78-488e-b000-ab63a2088d1e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:24.265307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4fc8d-ac78-488e-b000-ab63a2088d1e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:24.323746 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c477a6-1cca-4362-b9a3-6c5888a01cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:24.324349 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c477a6-1cca-4362-b9a3-6c5888a01cde admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.324623 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c477a6-1cca-4362-b9a3-6c5888a01cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.324889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c477a6-1cca-4362-b9a3-6c5888a01cde admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.325514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c477a6-1cca-4362-b9a3-6c5888a01cde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:24.327419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c477a6-1cca-4362-b9a3-6c5888a01cde admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.337317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:24.349031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1033/2056] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:24 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:24.353853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21556cd4-acdc-41cb-9857-dd713bba11f7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:24.363791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21556cd4-acdc-41cb-9857-dd713bba11f7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:24.369148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9888a6e-33e0-4337-bbde-fabe70a39064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:24.369740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9888a6e-33e0-4337-bbde-fabe70a39064 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.370020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9888a6e-33e0-4337-bbde-fabe70a39064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.370328 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9888a6e-33e0-4337-bbde-fabe70a39064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.370964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9888a6e-33e0-4337-bbde-fabe70a39064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:24.372537 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9888a6e-33e0-4337-bbde-fabe70a39064 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.383630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1024/2057] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:24.387456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c477a6-1cca-4362-b9a3-6c5888a01cde admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:24.398142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c477a6-1cca-4362-b9a3-6c5888a01cde admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:24.403050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a701b-37c3-459c-981a-f19dfc63907a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:24.403694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a701b-37c3-459c-981a-f19dfc63907a admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.403972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a701b-37c3-459c-981a-f19dfc63907a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.404244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a701b-37c3-459c-981a-f19dfc63907a admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.404884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a701b-37c3-459c-981a-f19dfc63907a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:24.406798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a701b-37c3-459c-981a-f19dfc63907a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.419146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1034/2058] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:24.423442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9888a6e-33e0-4337-bbde-fabe70a39064 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:24.434086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9888a6e-33e0-4337-bbde-fabe70a39064 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:24.438943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78031976-e2d7-49aa-880a-881817f1f4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:24.439587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78031976-e2d7-49aa-880a-881817f1f4ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.439865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78031976-e2d7-49aa-880a-881817f1f4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.440162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78031976-e2d7-49aa-880a-881817f1f4ef admin admin] PATH_INFO: `/v3/projects/c4b8f81571224fe4b54b05aa665615b2/users/83a6efe62f4f4aaea7f42ab88bd4b65f/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.455052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78031976-e2d7-49aa-880a-881817f1f4ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b8f81571224fe4b54b05aa665615b2, user_id=83a6efe62f4f4aaea7f42ab88bd4b65f, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:24.457693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78031976-e2d7-49aa-880a-881817f1f4ef admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.470797 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1025/2059] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:24 2022] PUT /identity/v3/projects/c4b8f81571224fe4b54b05aa665615b2/users/83a6efe62f4f4aaea7f42ab88bd4b65f/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:24.482233 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a898c-dc8d-4234-8406-9b8db4afadd6 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.482518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a898c-dc8d-4234-8406-9b8db4afadd6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.482790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a898c-dc8d-4234-8406-9b8db4afadd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.510578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c80a898c-dc8d-4234-8406-9b8db4afadd6 None None] MFA Rules not processed for user `83a6efe62f4f4aaea7f42ab88bd4b65f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:24.511777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c80a898c-dc8d-4234-8406-9b8db4afadd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:24.574495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1035/2060] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:50:24.579041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78031976-e2d7-49aa-880a-881817f1f4ef admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:24.605788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:24.641764 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78031976-e2d7-49aa-880a-881817f1f4ef admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:24.646941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536e490-46ae-4378-94ca-d4ff796f4d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:24.647617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e490-46ae-4378-94ca-d4ff796f4d17 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.647888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e490-46ae-4378-94ca-d4ff796f4d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.648159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e490-46ae-4378-94ca-d4ff796f4d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.648788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536e490-46ae-4378-94ca-d4ff796f4d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:24.650713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536e490-46ae-4378-94ca-d4ff796f4d17 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.676158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1026/2061] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:24 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:50:24.680230 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a898c-dc8d-4234-8406-9b8db4afadd6 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:24.690681 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a898c-dc8d-4234-8406-9b8db4afadd6 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:24.748557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd093f-59ae-4723-8e66-8380edf54604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:24.749164 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd093f-59ae-4723-8e66-8380edf54604 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.749442 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd093f-59ae-4723-8e66-8380edf54604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.749710 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd093f-59ae-4723-8e66-8380edf54604 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.750372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd093f-59ae-4723-8e66-8380edf54604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:24.752269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd093f-59ae-4723-8e66-8380edf54604 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.780065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1036/2062] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:24 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:24.784272 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536e490-46ae-4378-94ca-d4ff796f4d17 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:24.794940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536e490-46ae-4378-94ca-d4ff796f4d17 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:24.799516 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71e768-7822-4ce2-8833-3a1a9e0796e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:24.800104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71e768-7822-4ce2-8833-3a1a9e0796e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.800384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71e768-7822-4ce2-8833-3a1a9e0796e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.800654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71e768-7822-4ce2-8833-3a1a9e0796e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.801293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71e768-7822-4ce2-8833-3a1a9e0796e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:24.803462 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71e768-7822-4ce2-8833-3a1a9e0796e8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.814067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1027/2063] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:24.818778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd093f-59ae-4723-8e66-8380edf54604 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:24.828677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd093f-59ae-4723-8e66-8380edf54604 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:24.833217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053d240-b72f-44c2-bc50-cc2729e80ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:24.833899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053d240-b72f-44c2-bc50-cc2729e80ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.834381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053d240-b72f-44c2-bc50-cc2729e80ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.834654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053d240-b72f-44c2-bc50-cc2729e80ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.835289 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053d240-b72f-44c2-bc50-cc2729e80ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:24.836850 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053d240-b72f-44c2-bc50-cc2729e80ff8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.846864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1037/2064] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:50:24.850831 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71e768-7822-4ce2-8833-3a1a9e0796e8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:24.860433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71e768-7822-4ce2-8833-3a1a9e0796e8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:24.865541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c717d3c-f9da-4315-a25a-493cef8b5ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:24.866319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c717d3c-f9da-4315-a25a-493cef8b5ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.866604 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c717d3c-f9da-4315-a25a-493cef8b5ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.866875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c717d3c-f9da-4315-a25a-493cef8b5ff4 admin admin] PATH_INFO: `/v3/projects/1d08131c12db4ec0b4b4d7a3a3928606/users/3ac1f713dd7847d7b24ef65be06c94ba/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.881817 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c717d3c-f9da-4315-a25a-493cef8b5ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d08131c12db4ec0b4b4d7a3a3928606, user_id=3ac1f713dd7847d7b24ef65be06c94ba, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:24.884496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c717d3c-f9da-4315-a25a-493cef8b5ff4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:24.897414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1028/2065] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:24 2022] PUT /identity/v3/projects/1d08131c12db4ec0b4b4d7a3a3928606/users/3ac1f713dd7847d7b24ef65be06c94ba/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:24.907800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8902ac-3df4-43b1-a69f-a93e635e3fe0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:24.908186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8902ac-3df4-43b1-a69f-a93e635e3fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:24.908460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8902ac-3df4-43b1-a69f-a93e635e3fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:24.937341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-de8902ac-3df4-43b1-a69f-a93e635e3fe0 None None] MFA Rules not processed for user `3ac1f713dd7847d7b24ef65be06c94ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:24.938558 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-de8902ac-3df4-43b1-a69f-a93e635e3fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:24.979470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:25.003511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1038/2066] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:25.008073 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c717d3c-f9da-4315-a25a-493cef8b5ff4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:25.069647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c717d3c-f9da-4315-a25a-493cef8b5ff4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:25.074343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00234ae9-8a40-4342-928f-9d4fe1585291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:25.074941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00234ae9-8a40-4342-928f-9d4fe1585291 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:25.075221 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00234ae9-8a40-4342-928f-9d4fe1585291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:25.075493 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00234ae9-8a40-4342-928f-9d4fe1585291 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:25.076128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00234ae9-8a40-4342-928f-9d4fe1585291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:25.078017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00234ae9-8a40-4342-928f-9d4fe1585291 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:25.105355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1029/2067] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:25 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:25.109489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8902ac-3df4-43b1-a69f-a93e635e3fe0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:25.120340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8902ac-3df4-43b1-a69f-a93e635e3fe0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:25.176624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca7e8b-c127-4c07-9706-72c9fa96f402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:25.177222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca7e8b-c127-4c07-9706-72c9fa96f402 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:25.178029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca7e8b-c127-4c07-9706-72c9fa96f402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:25.178328 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca7e8b-c127-4c07-9706-72c9fa96f402 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:25.179043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca7e8b-c127-4c07-9706-72c9fa96f402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:25.180933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca7e8b-c127-4c07-9706-72c9fa96f402 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:25.203360 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1039/2068] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:25 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:25.208070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00234ae9-8a40-4342-928f-9d4fe1585291 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:25.218317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00234ae9-8a40-4342-928f-9d4fe1585291 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:25.222786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc895e-7301-4c4f-9a39-f40e1aa89eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:25.223369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc895e-7301-4c4f-9a39-f40e1aa89eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:25.223643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc895e-7301-4c4f-9a39-f40e1aa89eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:25.223911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc895e-7301-4c4f-9a39-f40e1aa89eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:25.225039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc895e-7301-4c4f-9a39-f40e1aa89eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:25.226667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc895e-7301-4c4f-9a39-f40e1aa89eb5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:25.236994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1030/2069] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:25.241813 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca7e8b-c127-4c07-9706-72c9fa96f402 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:25.251797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca7e8b-c127-4c07-9706-72c9fa96f402 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:25.256861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c442e8-9d55-4f41-89b1-4b625a24e300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:25.257510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c442e8-9d55-4f41-89b1-4b625a24e300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:25.257785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c442e8-9d55-4f41-89b1-4b625a24e300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:25.258056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c442e8-9d55-4f41-89b1-4b625a24e300 admin admin] PATH_INFO: `/v3/projects/2d6560a8ca2a4a1b8be7b12c8c9bb368/users/225a241f978a408daa02edae38b6e1d6/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:25.272805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c442e8-9d55-4f41-89b1-4b625a24e300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6560a8ca2a4a1b8be7b12c8c9bb368, user_id=225a241f978a408daa02edae38b6e1d6, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:25.275465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c442e8-9d55-4f41-89b1-4b625a24e300 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:25.288767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1040/2070] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:25 2022] PUT /identity/v3/projects/2d6560a8ca2a4a1b8be7b12c8c9bb368/users/225a241f978a408daa02edae38b6e1d6/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:25.292627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc895e-7301-4c4f-9a39-f40e1aa89eb5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:25.303013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc895e-7301-4c4f-9a39-f40e1aa89eb5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:25.346147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:25.359332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd3cbd-34de-4f8c-b2fd-15eaefdff0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:25.359921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd3cbd-34de-4f8c-b2fd-15eaefdff0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:25.360195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd3cbd-34de-4f8c-b2fd-15eaefdff0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:25.360462 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd3cbd-34de-4f8c-b2fd-15eaefdff0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:25.361089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd3cbd-34de-4f8c-b2fd-15eaefdff0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:25.362708 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd3cbd-34de-4f8c-b2fd-15eaefdff0ee admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:25.373035 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1031/2071] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:25.377089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c442e8-9d55-4f41-89b1-4b625a24e300 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:25.387741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c442e8-9d55-4f41-89b1-4b625a24e300 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:25.392237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031a80b-69b6-4c10-b542-e9a49db40452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:25.392873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031a80b-69b6-4c10-b542-e9a49db40452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:25.393148 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031a80b-69b6-4c10-b542-e9a49db40452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:25.393417 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031a80b-69b6-4c10-b542-e9a49db40452 admin admin] PATH_INFO: `/v3/projects/2d6560a8ca2a4a1b8be7b12c8c9bb368/users/225a241f978a408daa02edae38b6e1d6/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:25.395311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031a80b-69b6-4c10-b542-e9a49db40452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6560a8ca2a4a1b8be7b12c8c9bb368, user_id=225a241f978a408daa02edae38b6e1d6, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:25.398657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031a80b-69b6-4c10-b542-e9a49db40452 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:25.409441 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1041/2072] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:25 2022] PUT /identity/v3/projects/2d6560a8ca2a4a1b8be7b12c8c9bb368/users/225a241f978a408daa02edae38b6e1d6/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:25.420732 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97510-b83a-476f-8ec7-b253005ab29e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:25.421021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97510-b83a-476f-8ec7-b253005ab29e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:25.421307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c97510-b83a-476f-8ec7-b253005ab29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:25.451500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b6c97510-b83a-476f-8ec7-b253005ab29e None None] MFA Rules not processed for user `225a241f978a408daa02edae38b6e1d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:25.452727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b6c97510-b83a-476f-8ec7-b253005ab29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:25.521843 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1032/2073] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:25.532918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031a80b-69b6-4c10-b542-e9a49db40452 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:25.534698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1031a80b-69b6-4c10-b542-e9a49db40452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:25.592014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031a80b-69b6-4c10-b542-e9a49db40452 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:25.595785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e640477-77f4-4d62-ad57-4e5253ac617a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:25.596382 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e640477-77f4-4d62-ad57-4e5253ac617a service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:25.596657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e640477-77f4-4d62-ad57-4e5253ac617a service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:25.596923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e640477-77f4-4d62-ad57-4e5253ac617a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:25.641718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:25.654446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e640477-77f4-4d62-ad57-4e5253ac617a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:25.656158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e640477-77f4-4d62-ad57-4e5253ac617a service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:25.664991 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1042/2074] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:27.215195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c97510-b83a-476f-8ec7-b253005ab29e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:27.225859 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c97510-b83a-476f-8ec7-b253005ab29e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:27.229422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a8bc8-4e37-4899-bee5-5276d23ea701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:27.230019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a8bc8-4e37-4899-bee5-5276d23ea701 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:27.230488 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a8bc8-4e37-4899-bee5-5276d23ea701 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:27.230761 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a8bc8-4e37-4899-bee5-5276d23ea701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:27.288927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a8bc8-4e37-4899-bee5-5276d23ea701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:27.290373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a8bc8-4e37-4899-bee5-5276d23ea701 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:27.299853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1033/2075] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:27.850612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e640477-77f4-4d62-ad57-4e5253ac617a service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:27.859544 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e640477-77f4-4d62-ad57-4e5253ac617a service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:27.863345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa0ff3-77f2-45e3-ab4d-9209665c1fa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:27.863970 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa0ff3-77f2-45e3-ab4d-9209665c1fa0 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:27.864244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa0ff3-77f2-45e3-ab4d-9209665c1fa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:27.864512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa0ff3-77f2-45e3-ab4d-9209665c1fa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:27.873732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa0ff3-77f2-45e3-ab4d-9209665c1fa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:27.875166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa0ff3-77f2-45e3-ab4d-9209665c1fa0 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:27.885284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1043/2076] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:31.781111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a8bc8-4e37-4899-bee5-5276d23ea701 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:31.849075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a8bc8-4e37-4899-bee5-5276d23ea701 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:31.854283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:31.854911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:31.855190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:31.855459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] PATH_INFO: `/v3/users/83a6efe62f4f4aaea7f42ab88bd4b65f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:31.856566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83a6efe62f4f4aaea7f42ab88bd4b65f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:31.858500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:31.868691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:31.907568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83a6efe62f4f4aaea7f42ab88bd4b65f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:31.908205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83a6efe62f4f4aaea7f42ab88bd4b65f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:31.908810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83a6efe62f4f4aaea7f42ab88bd4b65f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:31.915731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83a6efe62f4f4aaea7f42ab88bd4b65f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:31.928962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83a6efe62f4f4aaea7f42ab88bd4b65f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:31.941357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1034/2077] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:31 2022] DELETE /identity/v3/users/83a6efe62f4f4aaea7f42ab88bd4b65f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:31.946898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401c94-6d99-4895-ae6b-d761df230a92 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:31.947414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401c94-6d99-4895-ae6b-d761df230a92 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:31.947510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401c94-6d99-4895-ae6b-d761df230a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:31.968737 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b7401c94-6d99-4895-ae6b-d761df230a92 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:31.970080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b7401c94-6d99-4895-ae6b-d761df230a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:32.030281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1044/2078] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:32.040565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:32.042062 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:32.101666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90736d3-7837-4785-91d6-5b0029295e12 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:32.106660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd94ff-1c6c-419f-a24d-f294f7d837a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:32.107290 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd94ff-1c6c-419f-a24d-f294f7d837a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:32.107572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd94ff-1c6c-419f-a24d-f294f7d837a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:32.107842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd94ff-1c6c-419f-a24d-f294f7d837a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:32.124414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd94ff-1c6c-419f-a24d-f294f7d837a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:32.126035 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd94ff-1c6c-419f-a24d-f294f7d837a2 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:32.137544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1035/2079] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:32.531081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7401c94-6d99-4895-ae6b-d761df230a92 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:32.532732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b7401c94-6d99-4895-ae6b-d761df230a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:32.566463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7401c94-6d99-4895-ae6b-d761df230a92 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:32.574523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:32.575273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:32.575610 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:32.576058 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] PATH_INFO: `/v3/projects/c4b8f81571224fe4b54b05aa665615b2` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:32.577185 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b8f81571224fe4b54b05aa665615b2)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:32.581243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:32.634352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b8f81571224fe4b54b05aa665615b2', 'request_id': 'req-8dc3db7b-3662-40cd-b9dc-9a3ade823076'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:32.640164 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b8f81571224fe4b54b05aa665615b2', 'request_id': 'req-8dc3db7b-3662-40cd-b9dc-9a3ade823076'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:32.648526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b8f81571224fe4b54b05aa665615b2', 'request_id': 'req-8dc3db7b-3662-40cd-b9dc-9a3ade823076'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:32.660516 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] The token cache is being invalidate because project c4b8f81571224fe4b54b05aa665615b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:32.660800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:32.661504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:32.663310 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1045/2080] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:32 2022] DELETE /identity/v3/projects/c4b8f81571224fe4b54b05aa665615b2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:32.667375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd94ff-1c6c-419f-a24d-f294f7d837a2 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:32.677076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd94ff-1c6c-419f-a24d-f294f7d837a2 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:32.735880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:32.736519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:32.736790 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:32.737139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] PATH_INFO: `/v3/users/3ac1f713dd7847d7b24ef65be06c94ba` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:32.738312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac1f713dd7847d7b24ef65be06c94ba)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:32.740762 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:32.778522 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:32.792786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac1f713dd7847d7b24ef65be06c94ba'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:32.793437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac1f713dd7847d7b24ef65be06c94ba'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:32.794038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac1f713dd7847d7b24ef65be06c94ba'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:32.801355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac1f713dd7847d7b24ef65be06c94ba'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:32.813825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac1f713dd7847d7b24ef65be06c94ba'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:32.828590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1036/2081] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:32 2022] DELETE /identity/v3/users/3ac1f713dd7847d7b24ef65be06c94ba => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:33.116678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:33.118777 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:33.143081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3db7b-3662-40cd-b9dc-9a3ade823076 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:33.207248 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:33.217194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:33.217958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:33.218292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:33.218769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] PATH_INFO: `/v3/projects/1d08131c12db4ec0b4b4d7a3a3928606` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:33.221090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d08131c12db4ec0b4b4d7a3a3928606)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:33.223532 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:33.279985 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d08131c12db4ec0b4b4d7a3a3928606', 'request_id': 'req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.285905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d08131c12db4ec0b4b4d7a3a3928606', 'request_id': 'req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.293732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d08131c12db4ec0b4b4d7a3a3928606', 'request_id': 'req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.305341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] The token cache is being invalidate because project 1d08131c12db4ec0b4b4d7a3a3928606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:33.305628 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.306289 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.307773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1046/2082] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:33 2022] DELETE /identity/v3/projects/1d08131c12db4ec0b4b4d7a3a3928606 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:33.312218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:33.313943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:33.328667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e1f68-7f8d-4cb0-b4ee-fdec1c866ded admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:33.389910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c776d-fb25-4c8c-a508-e518615d1839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:33.390744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c776d-fb25-4c8c-a508-e518615d1839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:33.391160 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c776d-fb25-4c8c-a508-e518615d1839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:33.391532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c776d-fb25-4c8c-a508-e518615d1839 admin admin] PATH_INFO: `/v3/users/225a241f978a408daa02edae38b6e1d6` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:33.392790 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c776d-fb25-4c8c-a508-e518615d1839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=225a241f978a408daa02edae38b6e1d6)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:33.394781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c776d-fb25-4c8c-a508-e518615d1839 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:33.445906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-311c776d-fb25-4c8c-a508-e518615d1839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '225a241f978a408daa02edae38b6e1d6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.446539 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-311c776d-fb25-4c8c-a508-e518615d1839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '225a241f978a408daa02edae38b6e1d6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.447345 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-311c776d-fb25-4c8c-a508-e518615d1839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '225a241f978a408daa02edae38b6e1d6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.456721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-311c776d-fb25-4c8c-a508-e518615d1839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '225a241f978a408daa02edae38b6e1d6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.472392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-311c776d-fb25-4c8c-a508-e518615d1839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '225a241f978a408daa02edae38b6e1d6'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.486576 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1037/2083] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:33 2022] DELETE /identity/v3/users/225a241f978a408daa02edae38b6e1d6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:33.776133 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:33.777743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:33.791107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8ac71-8b97-4ce3-831e-30fa1b2b0810 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:33.851918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:33.852554 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:33.852831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:33.853103 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] PATH_INFO: `/v3/projects/2d6560a8ca2a4a1b8be7b12c8c9bb368` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:33.854232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6560a8ca2a4a1b8be7b12c8c9bb368)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:33.856128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:33.906951 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6560a8ca2a4a1b8be7b12c8c9bb368', 'request_id': 'req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.912998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6560a8ca2a4a1b8be7b12c8c9bb368', 'request_id': 'req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.923141 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6560a8ca2a4a1b8be7b12c8c9bb368', 'request_id': 'req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.935610 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] The token cache is being invalidate because project 2d6560a8ca2a4a1b8be7b12c8c9bb368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:33.935900 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.936547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9a04c07a-b2b4-4059-92fd-6eba8d3c4368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:33.938112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1047/2084] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:33 2022] DELETE /identity/v3/projects/2d6560a8ca2a4a1b8be7b12c8c9bb368 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:33.943550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac15c60-100a-4494-88b2-03df8afc493d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:33.944038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac15c60-100a-4494-88b2-03df8afc493d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:33.944185 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac15c60-100a-4494-88b2-03df8afc493d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:33.966331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2ac15c60-100a-4494-88b2-03df8afc493d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:33.967540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2ac15c60-100a-4494-88b2-03df8afc493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:33.992875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:34.041778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1038/2085] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:34.043618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9fffa-c03c-443d-b77a-255b5e78ac49 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.043979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9fffa-c03c-443d-b77a-255b5e78ac49 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.044314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9fffa-c03c-443d-b77a-255b5e78ac49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.065177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-67e9fffa-c03c-443d-b77a-255b5e78ac49 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:34.066512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-67e9fffa-c03c-443d-b77a-255b5e78ac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:34.075716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1048/2086] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:34.080131 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac15c60-100a-4494-88b2-03df8afc493d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:34.091743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac15c60-100a-4494-88b2-03df8afc493d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:34.096531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb68309-0738-4fdd-83ac-c905b73d3b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:34.097145 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb68309-0738-4fdd-83ac-c905b73d3b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.097425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb68309-0738-4fdd-83ac-c905b73d3b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.097781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb68309-0738-4fdd-83ac-c905b73d3b57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.098365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb68309-0738-4fdd-83ac-c905b73d3b57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:50:34.098669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb68309-0738-4fdd-83ac-c905b73d3b57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:34.100342 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb68309-0738-4fdd-83ac-c905b73d3b57 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:34.112513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1039/2087] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:50:34.116948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9fffa-c03c-443d-b77a-255b5e78ac49 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:34.127177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9fffa-c03c-443d-b77a-255b5e78ac49 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:34.132593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf9227-0e19-447b-9197-88cf160913ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:34.133194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf9227-0e19-447b-9197-88cf160913ca admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.133471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf9227-0e19-447b-9197-88cf160913ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.133742 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf9227-0e19-447b-9197-88cf160913ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.134411 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf9227-0e19-447b-9197-88cf160913ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:34.136331 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf9227-0e19-447b-9197-88cf160913ca admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:34.152547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:34.174409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1049/2088] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:34 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:34.175676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb68309-0738-4fdd-83ac-c905b73d3b57 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:34.186104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb68309-0738-4fdd-83ac-c905b73d3b57 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:34.253457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9856a-3903-45e4-ab04-15e48dc0857f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:34.254068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9856a-3903-45e4-ab04-15e48dc0857f admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.254381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9856a-3903-45e4-ab04-15e48dc0857f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.254655 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9856a-3903-45e4-ab04-15e48dc0857f admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.255840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9856a-3903-45e4-ab04-15e48dc0857f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:34.257767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9856a-3903-45e4-ab04-15e48dc0857f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:34.283158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1040/2089] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:34 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:50:34.284084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf9227-0e19-447b-9197-88cf160913ca admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:34.296203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf9227-0e19-447b-9197-88cf160913ca admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:34.301140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507775c-7080-456f-9ccf-7147b567e708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:34.301837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507775c-7080-456f-9ccf-7147b567e708 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.302749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507775c-7080-456f-9ccf-7147b567e708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.303142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507775c-7080-456f-9ccf-7147b567e708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.303944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507775c-7080-456f-9ccf-7147b567e708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:34.305612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507775c-7080-456f-9ccf-7147b567e708 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:34.318962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1050/2090] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:50:34.319101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729643d-2150-4c29-9eb5-b5492c8e4667 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.319101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729643d-2150-4c29-9eb5-b5492c8e4667 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.319101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729643d-2150-4c29-9eb5-b5492c8e4667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.321940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507775c-7080-456f-9ccf-7147b567e708 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:34.331933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507775c-7080-456f-9ccf-7147b567e708 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:34.337237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cee902-d86a-4738-a324-e40ccb716bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:34.337826 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cee902-d86a-4738-a324-e40ccb716bef admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.338102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cee902-d86a-4738-a324-e40ccb716bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.338398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cee902-d86a-4738-a324-e40ccb716bef admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.338718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2729643d-2150-4c29-9eb5-b5492c8e4667 None None] MFA Rules not processed for user `1461a94dd8ef48f79a0d8642a1bc1e58`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:34.339074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cee902-d86a-4738-a324-e40ccb716bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:34.339909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2729643d-2150-4c29-9eb5-b5492c8e4667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:34.340620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cee902-d86a-4738-a324-e40ccb716bef admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:34.350559 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1051/2091] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:50:34.354531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cee902-d86a-4738-a324-e40ccb716bef admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:34.363917 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cee902-d86a-4738-a324-e40ccb716bef admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:34.369072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffffa3-0886-4582-9dd1-681b8a40a110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:34.369697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffffa3-0886-4582-9dd1-681b8a40a110 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.369969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffffa3-0886-4582-9dd1-681b8a40a110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.370268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffffa3-0886-4582-9dd1-681b8a40a110 admin admin] PATH_INFO: `/v3/projects/7e0c52eb85b445efb60bc523caadf775/users/078b9ac52ead457489bedb268c6fb766/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.385600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffffa3-0886-4582-9dd1-681b8a40a110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0c52eb85b445efb60bc523caadf775, user_id=078b9ac52ead457489bedb268c6fb766, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:34.388240 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffffa3-0886-4582-9dd1-681b8a40a110 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:34.395938 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1041/2092] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:34.401459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1052/2093] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:34 2022] PUT /identity/v3/projects/7e0c52eb85b445efb60bc523caadf775/users/078b9ac52ead457489bedb268c6fb766/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:50:34.412239 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71940f02-71ff-4764-b686-6049e666cfb6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.412669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71940f02-71ff-4764-b686-6049e666cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.413026 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71940f02-71ff-4764-b686-6049e666cfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.447202 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-71940f02-71ff-4764-b686-6049e666cfb6 None None] MFA Rules not processed for user `078b9ac52ead457489bedb268c6fb766`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:34.448455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-71940f02-71ff-4764-b686-6049e666cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:34.489611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:34.515159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1042/2094] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:34.519699 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffffa3-0886-4582-9dd1-681b8a40a110 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:34.582499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffffa3-0886-4582-9dd1-681b8a40a110 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:34.588485 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b094abb-3f4e-4a17-84bc-2caae8000cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:34.589441 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094abb-3f4e-4a17-84bc-2caae8000cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.589765 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094abb-3f4e-4a17-84bc-2caae8000cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.590000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094abb-3f4e-4a17-84bc-2caae8000cf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.590810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b094abb-3f4e-4a17-84bc-2caae8000cf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:34.592828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b094abb-3f4e-4a17-84bc-2caae8000cf8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:34.624321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1053/2095] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:34 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:34.625777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71940f02-71ff-4764-b686-6049e666cfb6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:34.637450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71940f02-71ff-4764-b686-6049e666cfb6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:34.694212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eec5ff-4853-40e1-bd0c-dae2e29b4ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:34.694834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eec5ff-4853-40e1-bd0c-dae2e29b4ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.695114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eec5ff-4853-40e1-bd0c-dae2e29b4ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.695385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eec5ff-4853-40e1-bd0c-dae2e29b4ca2 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.696018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eec5ff-4853-40e1-bd0c-dae2e29b4ca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:34.697926 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eec5ff-4853-40e1-bd0c-dae2e29b4ca2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:34.720520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1043/2096] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:34 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:34.725117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b094abb-3f4e-4a17-84bc-2caae8000cf8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:34.735848 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b094abb-3f4e-4a17-84bc-2caae8000cf8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:34.740725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cf32b-c4fa-4be4-a0f9-4adc19a32fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:34.741253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cf32b-c4fa-4be4-a0f9-4adc19a32fca admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.741530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cf32b-c4fa-4be4-a0f9-4adc19a32fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.741797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cf32b-c4fa-4be4-a0f9-4adc19a32fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.742462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cf32b-c4fa-4be4-a0f9-4adc19a32fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:34.744041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cf32b-c4fa-4be4-a0f9-4adc19a32fca admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:34.755144 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1054/2097] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:34.759694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eec5ff-4853-40e1-bd0c-dae2e29b4ca2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:34.769838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eec5ff-4853-40e1-bd0c-dae2e29b4ca2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:34.775142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bba902-f0eb-4349-bef8-13559000acf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:34.775741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bba902-f0eb-4349-bef8-13559000acf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.776019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bba902-f0eb-4349-bef8-13559000acf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.776290 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bba902-f0eb-4349-bef8-13559000acf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.776937 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bba902-f0eb-4349-bef8-13559000acf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:34.778671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bba902-f0eb-4349-bef8-13559000acf5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:34.789145 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1044/2098] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:50:34.793347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cf32b-c4fa-4be4-a0f9-4adc19a32fca admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:34.803092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cf32b-c4fa-4be4-a0f9-4adc19a32fca admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:34.807680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd456d9f-40d7-4a41-bc66-422aebf646d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:34.808320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456d9f-40d7-4a41-bc66-422aebf646d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.808596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456d9f-40d7-4a41-bc66-422aebf646d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.808860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456d9f-40d7-4a41-bc66-422aebf646d3 admin admin] PATH_INFO: `/v3/projects/2ac7fb71fb4740ec9976550e940f46a7/users/002311b535db4a599c391167bb8dd0e5/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.825495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd456d9f-40d7-4a41-bc66-422aebf646d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac7fb71fb4740ec9976550e940f46a7, user_id=002311b535db4a599c391167bb8dd0e5, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:34.828223 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd456d9f-40d7-4a41-bc66-422aebf646d3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:34.842475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1055/2099] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:34 2022] PUT /identity/v3/projects/2ac7fb71fb4740ec9976550e940f46a7/users/002311b535db4a599c391167bb8dd0e5/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:34.909558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01359-2eae-4f6e-97be-a8628fdb7f85 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:34.909839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01359-2eae-4f6e-97be-a8628fdb7f85 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:34.910120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01359-2eae-4f6e-97be-a8628fdb7f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:34.939152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a2a01359-2eae-4f6e-97be-a8628fdb7f85 None None] MFA Rules not processed for user `002311b535db4a599c391167bb8dd0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:34.940492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a2a01359-2eae-4f6e-97be-a8628fdb7f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:35.012717 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1045/2100] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:35.017081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd456d9f-40d7-4a41-bc66-422aebf646d3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:35.082800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:35.098760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd456d9f-40d7-4a41-bc66-422aebf646d3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:35.104538 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc14b2-2837-42d7-849a-148892a4aa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:35.105151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc14b2-2837-42d7-849a-148892a4aa1c admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:35.105444 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc14b2-2837-42d7-849a-148892a4aa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:35.105722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc14b2-2837-42d7-849a-148892a4aa1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:35.106388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc14b2-2837-42d7-849a-148892a4aa1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:35.108301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc14b2-2837-42d7-849a-148892a4aa1c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:35.141562 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1056/2101] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:35 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:35.142826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a01359-2eae-4f6e-97be-a8628fdb7f85 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:35.153652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a01359-2eae-4f6e-97be-a8628fdb7f85 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:35.201746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:35.218906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7693-3ac0-46c9-bac5-6a677de154c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:35.219712 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7693-3ac0-46c9-bac5-6a677de154c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:35.220004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7693-3ac0-46c9-bac5-6a677de154c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:35.220226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7693-3ac0-46c9-bac5-6a677de154c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:35.220926 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a7693-3ac0-46c9-bac5-6a677de154c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:35.223005 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a7693-3ac0-46c9-bac5-6a677de154c2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:35.246236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1046/2102] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:35 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:50:35.250110 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc14b2-2837-42d7-849a-148892a4aa1c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:35.261532 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc14b2-2837-42d7-849a-148892a4aa1c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:35.266125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72275a5-545b-423a-b6af-18e218c606c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:35.266767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72275a5-545b-423a-b6af-18e218c606c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:35.267048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72275a5-545b-423a-b6af-18e218c606c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:35.267344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72275a5-545b-423a-b6af-18e218c606c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:35.267985 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72275a5-545b-423a-b6af-18e218c606c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:35.269578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72275a5-545b-423a-b6af-18e218c606c0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:35.281577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1057/2103] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:35.285800 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7693-3ac0-46c9-bac5-6a677de154c2 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:35.296539 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7693-3ac0-46c9-bac5-6a677de154c2 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:35.301753 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f74ac2-3aad-4e05-8d61-e3d5c855cda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:35.303226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f74ac2-3aad-4e05-8d61-e3d5c855cda3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:35.303510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f74ac2-3aad-4e05-8d61-e3d5c855cda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:35.303788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f74ac2-3aad-4e05-8d61-e3d5c855cda3 admin admin] PATH_INFO: `/v3/projects/f24e85c402e74c569b0643f9876ab086/users/11f64c2a820c42e1ae0c7f592d34f3f8/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:35.321671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f74ac2-3aad-4e05-8d61-e3d5c855cda3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f24e85c402e74c569b0643f9876ab086, user_id=11f64c2a820c42e1ae0c7f592d34f3f8, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:35.324507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f74ac2-3aad-4e05-8d61-e3d5c855cda3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:35.338127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1047/2104] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:35 2022] PUT /identity/v3/projects/f24e85c402e74c569b0643f9876ab086/users/11f64c2a820c42e1ae0c7f592d34f3f8/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:35.342077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72275a5-545b-423a-b6af-18e218c606c0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:35.353061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72275a5-545b-423a-b6af-18e218c606c0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:35.415266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7d1f9-f196-42d7-bcfb-c618c93fad88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:35.415881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7d1f9-f196-42d7-bcfb-c618c93fad88 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:35.416162 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7d1f9-f196-42d7-bcfb-c618c93fad88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:35.416471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7d1f9-f196-42d7-bcfb-c618c93fad88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:35.417193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7d1f9-f196-42d7-bcfb-c618c93fad88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:35.418749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7d1f9-f196-42d7-bcfb-c618c93fad88 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:35.431285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1058/2105] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:35.435875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f74ac2-3aad-4e05-8d61-e3d5c855cda3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:35.446422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f74ac2-3aad-4e05-8d61-e3d5c855cda3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:35.451302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095de4c-ea06-47a6-9c5f-4a03585a8f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:35.451963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095de4c-ea06-47a6-9c5f-4a03585a8f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:35.452242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095de4c-ea06-47a6-9c5f-4a03585a8f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:35.452511 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095de4c-ea06-47a6-9c5f-4a03585a8f8f admin admin] PATH_INFO: `/v3/projects/f24e85c402e74c569b0643f9876ab086/users/11f64c2a820c42e1ae0c7f592d34f3f8/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:35.454556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095de4c-ea06-47a6-9c5f-4a03585a8f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f24e85c402e74c569b0643f9876ab086, user_id=11f64c2a820c42e1ae0c7f592d34f3f8, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:35.457186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095de4c-ea06-47a6-9c5f-4a03585a8f8f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:35.469308 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1048/2106] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:35 2022] PUT /identity/v3/projects/f24e85c402e74c569b0643f9876ab086/users/11f64c2a820c42e1ae0c7f592d34f3f8/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:35.480004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0f3a7-245d-4678-a339-cc7bd982393b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:35.480291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0f3a7-245d-4678-a339-cc7bd982393b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:35.480572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0f3a7-245d-4678-a339-cc7bd982393b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:35.509500 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-36d0f3a7-245d-4678-a339-cc7bd982393b None None] MFA Rules not processed for user `11f64c2a820c42e1ae0c7f592d34f3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:35.510719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-36d0f3a7-245d-4678-a339-cc7bd982393b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:35.584273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1059/2107] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:35.596947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095de4c-ea06-47a6-9c5f-4a03585a8f8f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:35.599125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3095de4c-ea06-47a6-9c5f-4a03585a8f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:35.658179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095de4c-ea06-47a6-9c5f-4a03585a8f8f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:35.661927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c318fb4-ba49-4341-bc0d-b3360a30429f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:35.662580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c318fb4-ba49-4341-bc0d-b3360a30429f service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:35.662997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c318fb4-ba49-4341-bc0d-b3360a30429f service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:35.663269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c318fb4-ba49-4341-bc0d-b3360a30429f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:35.720494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c318fb4-ba49-4341-bc0d-b3360a30429f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:35.721920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c318fb4-ba49-4341-bc0d-b3360a30429f service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:35.731542 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1049/2108] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:50:37.247613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0f3a7-245d-4678-a339-cc7bd982393b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:37.249137 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-36d0f3a7-245d-4678-a339-cc7bd982393b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:37.261642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0f3a7-245d-4678-a339-cc7bd982393b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:37.265633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01344ad2-ff25-4ec1-98ce-a03b382ddf1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:37.266282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01344ad2-ff25-4ec1-98ce-a03b382ddf1e service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:37.266562 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01344ad2-ff25-4ec1-98ce-a03b382ddf1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:37.266828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01344ad2-ff25-4ec1-98ce-a03b382ddf1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:37.317546 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:37.331173 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01344ad2-ff25-4ec1-98ce-a03b382ddf1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:37.332980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01344ad2-ff25-4ec1-98ce-a03b382ddf1e service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:37.342420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1060/2109] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:37.910113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c318fb4-ba49-4341-bc0d-b3360a30429f service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:37.918698 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c318fb4-ba49-4341-bc0d-b3360a30429f service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:37.922339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ff104-6a7f-4f9d-ae6e-70c9ff6e2901 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:37.922941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ff104-6a7f-4f9d-ae6e-70c9ff6e2901 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:37.923220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ff104-6a7f-4f9d-ae6e-70c9ff6e2901 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:37.923534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ff104-6a7f-4f9d-ae6e-70c9ff6e2901 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:37.933586 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ff104-6a7f-4f9d-ae6e-70c9ff6e2901 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:37.935102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ff104-6a7f-4f9d-ae6e-70c9ff6e2901 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:37.945474 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1050/2110] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:41.826434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01344ad2-ff25-4ec1-98ce-a03b382ddf1e service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:41.887361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01344ad2-ff25-4ec1-98ce-a03b382ddf1e service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:41.891840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:41.892456 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:41.892743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:41.893010 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] PATH_INFO: `/v3/users/078b9ac52ead457489bedb268c6fb766` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:41.894117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=078b9ac52ead457489bedb268c6fb766)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:41.896862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:41.942446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '078b9ac52ead457489bedb268c6fb766'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:41.944019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '078b9ac52ead457489bedb268c6fb766'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:41.955791 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '078b9ac52ead457489bedb268c6fb766'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:41.956478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '078b9ac52ead457489bedb268c6fb766'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:41.964322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '078b9ac52ead457489bedb268c6fb766'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:41.976908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1061/2111] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:41 2022] DELETE /identity/v3/users/078b9ac52ead457489bedb268c6fb766 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:41.981437 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2c055-b616-446d-b1a7-d1cca148cef5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:41.981933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2c055-b616-446d-b1a7-d1cca148cef5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:41.982040 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2c055-b616-446d-b1a7-d1cca148cef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:42.000591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-9ac2c055-b616-446d-b1a7-d1cca148cef5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:42.001769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9ac2c055-b616-446d-b1a7-d1cca148cef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:42.041865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:42.063952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1051/2112] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:42.075365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:42.077062 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:42.139160 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f1da3-173f-4c67-9457-62a977c72bef admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:42.143105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e569202-1c1e-48f4-90df-95c8a2265bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:42.143719 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e569202-1c1e-48f4-90df-95c8a2265bfb service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:42.144000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e569202-1c1e-48f4-90df-95c8a2265bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:42.144269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e569202-1c1e-48f4-90df-95c8a2265bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:42.154481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e569202-1c1e-48f4-90df-95c8a2265bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:42.156159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e569202-1c1e-48f4-90df-95c8a2265bfb service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:42.167229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1062/2113] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:42.459510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2c055-b616-446d-b1a7-d1cca148cef5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:42.461072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-9ac2c055-b616-446d-b1a7-d1cca148cef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:42.476011 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2c055-b616-446d-b1a7-d1cca148cef5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:42.482517 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:42.483177 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:42.483450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:42.483766 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] PATH_INFO: `/v3/projects/7e0c52eb85b445efb60bc523caadf775` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:42.484875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e0c52eb85b445efb60bc523caadf775)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:42.486794 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:42.535836 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e0c52eb85b445efb60bc523caadf775', 'request_id': 'req-29f162fa-af0f-40ea-8747-b1beb0b1c006'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:42.543206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e0c52eb85b445efb60bc523caadf775', 'request_id': 'req-29f162fa-af0f-40ea-8747-b1beb0b1c006'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:42.549400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e0c52eb85b445efb60bc523caadf775', 'request_id': 'req-29f162fa-af0f-40ea-8747-b1beb0b1c006'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:42.559869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] The token cache is being invalidate because project 7e0c52eb85b445efb60bc523caadf775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:42.560152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:42.560779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:42.562318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1052/2114] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:42 2022] DELETE /identity/v3/projects/7e0c52eb85b445efb60bc523caadf775 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:42.566380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e569202-1c1e-48f4-90df-95c8a2265bfb service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:42.576827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e569202-1c1e-48f4-90df-95c8a2265bfb service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:42.581184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:42.635243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:42.636057 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:42.636451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:42.636697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] PATH_INFO: `/v3/users/002311b535db4a599c391167bb8dd0e5` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:42.638107 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002311b535db4a599c391167bb8dd0e5)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:42.640088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:42.686449 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002311b535db4a599c391167bb8dd0e5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:42.687190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002311b535db4a599c391167bb8dd0e5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:42.700238 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002311b535db4a599c391167bb8dd0e5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:42.700973 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002311b535db4a599c391167bb8dd0e5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:42.708101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002311b535db4a599c391167bb8dd0e5'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:42.723123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1063/2115] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:42 2022] DELETE /identity/v3/users/002311b535db4a599c391167bb8dd0e5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:42.996320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:42.997827 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:43.011305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f162fa-af0f-40ea-8747-b1beb0b1c006 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:43.072369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:43.072999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:43.073303 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:43.073598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] PATH_INFO: `/v3/projects/2ac7fb71fb4740ec9976550e940f46a7` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:43.074746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac7fb71fb4740ec9976550e940f46a7)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:43.076666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:43.123316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac7fb71fb4740ec9976550e940f46a7', 'request_id': 'req-e3cf979c-21fa-417b-9106-1886b4e2ebe0'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.130894 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac7fb71fb4740ec9976550e940f46a7', 'request_id': 'req-e3cf979c-21fa-417b-9106-1886b4e2ebe0'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.137377 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac7fb71fb4740ec9976550e940f46a7', 'request_id': 'req-e3cf979c-21fa-417b-9106-1886b4e2ebe0'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.141953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:43.149885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] The token cache is being invalidate because project 2ac7fb71fb4740ec9976550e940f46a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:43.150373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.151118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.152751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1053/2116] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:42 2022] DELETE /identity/v3/projects/2ac7fb71fb4740ec9976550e940f46a7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:43.157059 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:43.158548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:43.171948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2f996-cf6d-4c20-9949-63a870376279 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:43.237098 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e08718-07b7-47ef-9cf3-e3e1d84c9384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:43.237869 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e08718-07b7-47ef-9cf3-e3e1d84c9384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:43.238242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e08718-07b7-47ef-9cf3-e3e1d84c9384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:43.238589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e08718-07b7-47ef-9cf3-e3e1d84c9384 admin admin] PATH_INFO: `/v3/users/11f64c2a820c42e1ae0c7f592d34f3f8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:43.239842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e08718-07b7-47ef-9cf3-e3e1d84c9384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f64c2a820c42e1ae0c7f592d34f3f8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:43.242054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e08718-07b7-47ef-9cf3-e3e1d84c9384 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:43.281242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:43.299253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a7e08718-07b7-47ef-9cf3-e3e1d84c9384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f64c2a820c42e1ae0c7f592d34f3f8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.299967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a7e08718-07b7-47ef-9cf3-e3e1d84c9384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f64c2a820c42e1ae0c7f592d34f3f8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.312603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a7e08718-07b7-47ef-9cf3-e3e1d84c9384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f64c2a820c42e1ae0c7f592d34f3f8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.313293 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a7e08718-07b7-47ef-9cf3-e3e1d84c9384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f64c2a820c42e1ae0c7f592d34f3f8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.321239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a7e08718-07b7-47ef-9cf3-e3e1d84c9384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f64c2a820c42e1ae0c7f592d34f3f8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.340353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1064/2117] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:50:43 2022] DELETE /identity/v3/users/11f64c2a820c42e1ae0c7f592d34f3f8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:50:43.607032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:43.608690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:43.622799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf979c-21fa-417b-9106-1886b4e2ebe0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:43.680043 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:43.680627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:43.680905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:43.681176 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] PATH_INFO: `/v3/projects/f24e85c402e74c569b0643f9876ab086` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:43.682342 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f24e85c402e74c569b0643f9876ab086)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:43.684235 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:43.729996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f24e85c402e74c569b0643f9876ab086', 'request_id': 'req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.736803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f24e85c402e74c569b0643f9876ab086', 'request_id': 'req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.743628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f24e85c402e74c569b0643f9876ab086', 'request_id': 'req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.754032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] The token cache is being invalidate because project f24e85c402e74c569b0643f9876ab086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:50:43.754352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.754976 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7d419d7c-35ff-4133-bc5d-0c469b1c8aad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:50:43.756968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1054/2118] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:50:43 2022] DELETE /identity/v3/projects/f24e85c402e74c569b0643f9876ab086 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:43.762003 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8557b5c-9543-4c20-a087-3743af90c1f6 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:43.762322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8557b5c-9543-4c20-a087-3743af90c1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:43.762593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8557b5c-9543-4c20-a087-3743af90c1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:43.782071 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c8557b5c-9543-4c20-a087-3743af90c1f6 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:43.783430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c8557b5c-9543-4c20-a087-3743af90c1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:43.840950 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1065/2119] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:43.850124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eff390-90c6-4835-9a76-64b0e3e1d09a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:43.850435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eff390-90c6-4835-9a76-64b0e3e1d09a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:43.850710 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eff390-90c6-4835-9a76-64b0e3e1d09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:43.870548 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-46eff390-90c6-4835-9a76-64b0e3e1d09a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:43.871744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-46eff390-90c6-4835-9a76-64b0e3e1d09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:43.879523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1055/2120] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:43.884413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8557b5c-9543-4c20-a087-3743af90c1f6 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:43.893987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8557b5c-9543-4c20-a087-3743af90c1f6 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:43.898661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f6c4e-3c2a-4188-93bd-8728672adf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:43.899247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f6c4e-3c2a-4188-93bd-8728672adf6f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:43.899523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f6c4e-3c2a-4188-93bd-8728672adf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:43.899793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f6c4e-3c2a-4188-93bd-8728672adf6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:43.900433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f6c4e-3c2a-4188-93bd-8728672adf6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:50:43.900734 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f6c4e-3c2a-4188-93bd-8728672adf6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:43.902336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f6c4e-3c2a-4188-93bd-8728672adf6f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:43.917130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1066/2121] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:50:43.918827 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eff390-90c6-4835-9a76-64b0e3e1d09a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:43.928749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eff390-90c6-4835-9a76-64b0e3e1d09a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:43.933397 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a7b26-c3fa-4535-8d5d-8dcbd7da078c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:43.933925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a7b26-c3fa-4535-8d5d-8dcbd7da078c admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:43.934225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a7b26-c3fa-4535-8d5d-8dcbd7da078c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:43.934494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a7b26-c3fa-4535-8d5d-8dcbd7da078c admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:43.935118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a7b26-c3fa-4535-8d5d-8dcbd7da078c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:43.936981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a7b26-c3fa-4535-8d5d-8dcbd7da078c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:43.965574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1056/2122] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:43 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:50:43.969819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f6c4e-3c2a-4188-93bd-8728672adf6f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:43.980608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f6c4e-3c2a-4188-93bd-8728672adf6f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:44.039156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cfdea-bd96-48d3-ab75-761a360c98a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:44.039770 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cfdea-bd96-48d3-ab75-761a360c98a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:44.040055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cfdea-bd96-48d3-ab75-761a360c98a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:44.040321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cfdea-bd96-48d3-ab75-761a360c98a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:44.040959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cfdea-bd96-48d3-ab75-761a360c98a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:44.042916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cfdea-bd96-48d3-ab75-761a360c98a3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:44.064792 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1067/2123] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:43 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:44.069072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a7b26-c3fa-4535-8d5d-8dcbd7da078c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:44.079869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a7b26-c3fa-4535-8d5d-8dcbd7da078c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:44.084710 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824d6f4-0b16-4c17-836d-558be7441364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:44.085309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824d6f4-0b16-4c17-836d-558be7441364 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:44.085587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824d6f4-0b16-4c17-836d-558be7441364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:44.085858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824d6f4-0b16-4c17-836d-558be7441364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:44.086538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824d6f4-0b16-4c17-836d-558be7441364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:44.088119 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824d6f4-0b16-4c17-836d-558be7441364 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:44.102156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1057/2124] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:50:44.103716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cfdea-bd96-48d3-ab75-761a360c98a3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:44.113620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cfdea-bd96-48d3-ab75-761a360c98a3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:44.120429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a488099-f807-4e57-92ed-eed225f6cf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:44.121157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a488099-f807-4e57-92ed-eed225f6cf08 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:44.121439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a488099-f807-4e57-92ed-eed225f6cf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:44.121828 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a488099-f807-4e57-92ed-eed225f6cf08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:44.122429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a488099-f807-4e57-92ed-eed225f6cf08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:44.124025 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a488099-f807-4e57-92ed-eed225f6cf08 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:44.135790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1068/2125] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:44.139852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824d6f4-0b16-4c17-836d-558be7441364 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:44.150894 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824d6f4-0b16-4c17-836d-558be7441364 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:44.164493 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00609c1-4520-4025-b579-0d87e68545a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:44.165266 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00609c1-4520-4025-b579-0d87e68545a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:44.165551 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00609c1-4520-4025-b579-0d87e68545a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:44.165890 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00609c1-4520-4025-b579-0d87e68545a1 admin admin] PATH_INFO: `/v3/projects/5db8c3fdabf94498948055239da40ca0/users/568d031158744101a6bb339e51149ed2/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:44.182003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00609c1-4520-4025-b579-0d87e68545a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db8c3fdabf94498948055239da40ca0, user_id=568d031158744101a6bb339e51149ed2, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:44.184809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00609c1-4520-4025-b579-0d87e68545a1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:44.200600 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1058/2126] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:44 2022] PUT /identity/v3/projects/5db8c3fdabf94498948055239da40ca0/users/568d031158744101a6bb339e51149ed2/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:44.205959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55340d6a-ba07-470a-b9ab-6bc9ff7d6b7c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:44.206537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55340d6a-ba07-470a-b9ab-6bc9ff7d6b7c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:44.206805 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55340d6a-ba07-470a-b9ab-6bc9ff7d6b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:44.228009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-55340d6a-ba07-470a-b9ab-6bc9ff7d6b7c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:44.229254 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-55340d6a-ba07-470a-b9ab-6bc9ff7d6b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:44.238982 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:44.296212 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1069/2127] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:44.306404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00609c1-4520-4025-b579-0d87e68545a1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:44.308100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a00609c1-4520-4025-b579-0d87e68545a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:44.335198 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:44.371108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00609c1-4520-4025-b579-0d87e68545a1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:44.374887 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb95b9-fe8e-4141-8da6-6e0e4badd4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:44.375544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb95b9-fe8e-4141-8da6-6e0e4badd4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:44.375905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb95b9-fe8e-4141-8da6-6e0e4badd4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:44.376089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb95b9-fe8e-4141-8da6-6e0e4badd4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:44.391558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb95b9-fe8e-4141-8da6-6e0e4badd4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:44.393416 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb95b9-fe8e-4141-8da6-6e0e4badd4c1 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:44.412020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1059/2128] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:50:49.291289 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603fd51-2640-499f-99a5-4a8a3c97223e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:49.291809 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603fd51-2640-499f-99a5-4a8a3c97223e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:49.291913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603fd51-2640-499f-99a5-4a8a3c97223e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:49.325875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1603fd51-2640-499f-99a5-4a8a3c97223e None None] MFA Rules not processed for user `568d031158744101a6bb339e51149ed2`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:49.327678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1603fd51-2640-499f-99a5-4a8a3c97223e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:49.488076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1070/2129] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:49.492942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb95b9-fe8e-4141-8da6-6e0e4badd4c1 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:49.505405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb95b9-fe8e-4141-8da6-6e0e4badd4c1 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:49.518188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c64b7c-3a8d-4ae9-857e-ed47d11acbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:49.518910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c64b7c-3a8d-4ae9-857e-ed47d11acbb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:49.519260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c64b7c-3a8d-4ae9-857e-ed47d11acbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:49.519634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c64b7c-3a8d-4ae9-857e-ed47d11acbb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:49.520481 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c64b7c-3a8d-4ae9-857e-ed47d11acbb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:49.526461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c64b7c-3a8d-4ae9-857e-ed47d11acbb0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:49.574764 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1060/2130] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:50:49 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:49.578000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603fd51-2640-499f-99a5-4a8a3c97223e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:49.591434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603fd51-2640-499f-99a5-4a8a3c97223e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:49.671847 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2b376-c093-4959-b8d8-9ff048b4130a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:49.672473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b376-c093-4959-b8d8-9ff048b4130a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:49.672760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b376-c093-4959-b8d8-9ff048b4130a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:49.673027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b376-c093-4959-b8d8-9ff048b4130a admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:49.673821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2b376-c093-4959-b8d8-9ff048b4130a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:49.675913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2b376-c093-4959-b8d8-9ff048b4130a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:49.704990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1071/2131] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:50:49 2022] POST /identity/v3/users => generated 295 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:50:49.710136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c64b7c-3a8d-4ae9-857e-ed47d11acbb0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:49.722028 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c64b7c-3a8d-4ae9-857e-ed47d11acbb0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:49.727772 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f93a33-3586-4d3b-9f23-55ed47ccb1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:49.728488 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f93a33-3586-4d3b-9f23-55ed47ccb1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:49.728806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f93a33-3586-4d3b-9f23-55ed47ccb1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:49.729137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f93a33-3586-4d3b-9f23-55ed47ccb1a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:49.729880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f93a33-3586-4d3b-9f23-55ed47ccb1a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:49.731709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f93a33-3586-4d3b-9f23-55ed47ccb1a1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:49.744884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1061/2132] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:50:49.749290 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2b376-c093-4959-b8d8-9ff048b4130a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:49.761068 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2b376-c093-4959-b8d8-9ff048b4130a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:49.767517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d0910-83ba-4be0-aa56-ed7c68d1a110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:49.768201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d0910-83ba-4be0-aa56-ed7c68d1a110 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:49.768473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d0910-83ba-4be0-aa56-ed7c68d1a110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:49.768917 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d0910-83ba-4be0-aa56-ed7c68d1a110 admin admin] PATH_INFO: `/v3/projects/e90aa35ce1724dd78e026c9048e03751/users/ed31432d45e348028027d30eb4d47b20/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:49.788487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d0910-83ba-4be0-aa56-ed7c68d1a110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e90aa35ce1724dd78e026c9048e03751, user_id=ed31432d45e348028027d30eb4d47b20, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:49.791224 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d0910-83ba-4be0-aa56-ed7c68d1a110 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:49.809307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1072/2133] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:49 2022] PUT /identity/v3/projects/e90aa35ce1724dd78e026c9048e03751/users/ed31432d45e348028027d30eb4d47b20/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:50:49.813172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f93a33-3586-4d3b-9f23-55ed47ccb1a1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:49.824245 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f93a33-3586-4d3b-9f23-55ed47ccb1a1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:49.892874 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709d456-8026-41b0-be2c-4cc2cf41e4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:49.893483 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709d456-8026-41b0-be2c-4cc2cf41e4dd admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:49.893765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709d456-8026-41b0-be2c-4cc2cf41e4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:49.894037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709d456-8026-41b0-be2c-4cc2cf41e4dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:49.894831 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709d456-8026-41b0-be2c-4cc2cf41e4dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:49.897333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709d456-8026-41b0-be2c-4cc2cf41e4dd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:49.907405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1062/2134] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:50:49.912241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d0910-83ba-4be0-aa56-ed7c68d1a110 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:49.923122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d0910-83ba-4be0-aa56-ed7c68d1a110 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:49.927874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd5b2f-eccd-40b2-83fa-c872576157d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:49.928591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd5b2f-eccd-40b2-83fa-c872576157d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:49.928935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd5b2f-eccd-40b2-83fa-c872576157d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:49.929271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd5b2f-eccd-40b2-83fa-c872576157d0 admin admin] PATH_INFO: `/v3/projects/e90aa35ce1724dd78e026c9048e03751/users/ed31432d45e348028027d30eb4d47b20/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:49.931237 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd5b2f-eccd-40b2-83fa-c872576157d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e90aa35ce1724dd78e026c9048e03751, user_id=ed31432d45e348028027d30eb4d47b20, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:49.933906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd5b2f-eccd-40b2-83fa-c872576157d0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:49.937590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:50:49.948698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1073/2135] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:50:49 2022] PUT /identity/v3/projects/e90aa35ce1724dd78e026c9048e03751/users/ed31432d45e348028027d30eb4d47b20/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:50:54.105023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c837b-2ebd-4a21-8d39-2c55623c220a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:54.105469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c837b-2ebd-4a21-8d39-2c55623c220a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:54.105570 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c837b-2ebd-4a21-8d39-2c55623c220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:54.135421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a47c837b-2ebd-4a21-8d39-2c55623c220a None None] MFA Rules not processed for user `ed31432d45e348028027d30eb4d47b20`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:50:54.136655 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a47c837b-2ebd-4a21-8d39-2c55623c220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:54.226346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1063/2136] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:50:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:50:54.235315 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd5b2f-eccd-40b2-83fa-c872576157d0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:50:54.237079 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d3fd5b2f-eccd-40b2-83fa-c872576157d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:50:54.314488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd5b2f-eccd-40b2-83fa-c872576157d0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:50:54.320738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de31e4-8520-4766-9578-99c4c749eeb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:50:54.321341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de31e4-8520-4766-9578-99c4c749eeb0 service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:50:54.321617 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de31e4-8520-4766-9578-99c4c749eeb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:50:54.321880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de31e4-8520-4766-9578-99c4c749eeb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:50:54.379209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de31e4-8520-4766-9578-99c4c749eeb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:50:54.381795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de31e4-8520-4766-9578-99c4c749eeb0 service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:50:54.390906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1074/2137] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:50:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:51:04.216394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:51:04.386755 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=58 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:51:18.167918 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612bd47-4081-4e10-9ab2-1af5138ff076 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:51:18.168462 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612bd47-4081-4e10-9ab2-1af5138ff076 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:51:18.168462 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612bd47-4081-4e10-9ab2-1af5138ff076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:51:18.187307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c612bd47-4081-4e10-9ab2-1af5138ff076 None None] MFA Rules not processed for user `568d031158744101a6bb339e51149ed2`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:51:18.188489 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c612bd47-4081-4e10-9ab2-1af5138ff076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:51:18.195336 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1064/2138] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:51:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:51:28.181863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:51:35.619552 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de31e4-8520-4766-9578-99c4c749eeb0 service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:51:35.621142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-61de31e4-8520-4766-9578-99c4c749eeb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:51:35.676931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de31e4-8520-4766-9578-99c4c749eeb0 service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:51:35.680457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dba18-1b80-4997-87f0-652b96ced858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:51:35.681045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dba18-1b80-4997-87f0-652b96ced858 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:51:35.681351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dba18-1b80-4997-87f0-652b96ced858 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:51:35.681616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dba18-1b80-4997-87f0-652b96ced858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:51:35.690528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0dba18-1b80-4997-87f0-652b96ced858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:51:35.691921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0dba18-1b80-4997-87f0-652b96ced858 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:51:35.701705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1075/2139] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:51:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:51:45.372035 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612bd47-4081-4e10-9ab2-1af5138ff076 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:51:45.381799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612bd47-4081-4e10-9ab2-1af5138ff076 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:51:45.385316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0764548-04f7-474b-b221-c929386ec768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:51:45.385903 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0764548-04f7-474b-b221-c929386ec768 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:51:45.386282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0764548-04f7-474b-b221-c929386ec768 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:51:45.386556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0764548-04f7-474b-b221-c929386ec768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:51:45.388270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f0764548-04f7-474b-b221-c929386ec768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:51:45.401161 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0764548-04f7-474b-b221-c929386ec768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:51:45.403110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0764548-04f7-474b-b221-c929386ec768 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:51:45.413847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1065/2140] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:51:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:51:45.697319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=26 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:51:47.790684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dba18-1b80-4997-87f0-652b96ced858 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:51:47.799528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dba18-1b80-4997-87f0-652b96ced858 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:51:47.803150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453ca3e-5579-43bd-8267-bf658a3c1e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:51:47.803735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ca3e-5579-43bd-8267-bf658a3c1e4e service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:51:47.804011 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ca3e-5579-43bd-8267-bf658a3c1e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:51:47.804277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453ca3e-5579-43bd-8267-bf658a3c1e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:51:47.805977 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b453ca3e-5579-43bd-8267-bf658a3c1e4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:51:47.819595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453ca3e-5579-43bd-8267-bf658a3c1e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:51:47.820992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453ca3e-5579-43bd-8267-bf658a3c1e4e service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:51:47.832495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1076/2141] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:51:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:51:52.927297 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0165178-e630-432a-b468-64e188429763 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:51:52.927837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0165178-e630-432a-b468-64e188429763 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:51:52.927837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0165178-e630-432a-b468-64e188429763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:51:52.957178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a0165178-e630-432a-b468-64e188429763 None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:51:52.958383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a0165178-e630-432a-b468-64e188429763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:51:53.014511 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1066/2142] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:51:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:51:53.061954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453ca3e-5579-43bd-8267-bf658a3c1e4e service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:51:53.063395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b453ca3e-5579-43bd-8267-bf658a3c1e4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:51:53.131434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453ca3e-5579-43bd-8267-bf658a3c1e4e service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:51:53.136312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a8c64-184f-41ed-b986-5b40cca15dce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:51:53.136913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a8c64-184f-41ed-b986-5b40cca15dce service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:51:53.137191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a8c64-184f-41ed-b986-5b40cca15dce service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:51:53.137990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a8c64-184f-41ed-b986-5b40cca15dce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:51:53.146037 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a8c64-184f-41ed-b986-5b40cca15dce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:51:53.147647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a8c64-184f-41ed-b986-5b40cca15dce service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:51:53.156554 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1077/2143] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:51:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:52:03.018937 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=35 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:03.152365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=38 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:07.202142 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0165178-e630-432a-b468-64e188429763 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:07.212233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0165178-e630-432a-b468-64e188429763 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:07.216051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283d76b-2f33-493d-ba57-24e92bec9de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:07.216673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283d76b-2f33-493d-ba57-24e92bec9de1 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:07.216944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283d76b-2f33-493d-ba57-24e92bec9de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:07.217213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283d76b-2f33-493d-ba57-24e92bec9de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:07.220349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0283d76b-2f33-493d-ba57-24e92bec9de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:07.287884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283d76b-2f33-493d-ba57-24e92bec9de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:07.289313 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283d76b-2f33-493d-ba57-24e92bec9de1 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:07.299566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1067/2144] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:52:10.766225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a8c64-184f-41ed-b986-5b40cca15dce service swift] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:10.768269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-027a8c64-184f-41ed-b986-5b40cca15dce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:10.786383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a8c64-184f-41ed-b986-5b40cca15dce service swift] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:10.791023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:10.791830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:10.792170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:10.792502 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] PATH_INFO: `/v3/users/1ea8feefb0614972898c1b1ad28077a8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:10.810075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea8feefb0614972898c1b1ad28077a8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:10.812319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:10.874906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea8feefb0614972898c1b1ad28077a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:10.875510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea8feefb0614972898c1b1ad28077a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:10.888936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eb39e-602a-4f30-b165-13c6b2d05442 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:10.888936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eb39e-602a-4f30-b165-13c6b2d05442 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:10.888936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eb39e-602a-4f30-b165-13c6b2d05442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:10.889640 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea8feefb0614972898c1b1ad28077a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:10.889844 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea8feefb0614972898c1b1ad28077a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:10.898039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea8feefb0614972898c1b1ad28077a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:10.912268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1078/2145] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:10 2022] DELETE /identity/v3/users/1ea8feefb0614972898c1b1ad28077a8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:10.946410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-be6eb39e-602a-4f30-b165-13c6b2d05442 None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:10.947622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-be6eb39e-602a-4f30-b165-13c6b2d05442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:11.020338 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1068/2146] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:52:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:11.040743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:11.042741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:11.141969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eb39e-602a-4f30-b165-13c6b2d05442 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:11.143440 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-be6eb39e-602a-4f30-b165-13c6b2d05442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:11.202209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37034294-f02c-42d4-b5d4-33962f4b00ea admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:11.216332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e49658-b384-4ea0-a492-db4a675aa7d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:11.217093 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e49658-b384-4ea0-a492-db4a675aa7d6 service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:11.217503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e49658-b384-4ea0-a492-db4a675aa7d6 service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:11.217840 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e49658-b384-4ea0-a492-db4a675aa7d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:11.227314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e49658-b384-4ea0-a492-db4a675aa7d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:11.229024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e49658-b384-4ea0-a492-db4a675aa7d6 service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:11.237954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1079/2147] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:52:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:52:11.245608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eb39e-602a-4f30-b165-13c6b2d05442 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:11.250972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:11.251767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:11.252108 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:11.252502 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] PATH_INFO: `/v3/projects/e0f08e43a5984a9398b3d5ec41e4282c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:11.263617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:11.266292 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f08e43a5984a9398b3d5ec41e4282c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:11.268718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:11.334073 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f08e43a5984a9398b3d5ec41e4282c', 'request_id': 'req-d9b479a0-7158-4a49-90d3-bf43bc69a974'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:11.348907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f08e43a5984a9398b3d5ec41e4282c', 'request_id': 'req-d9b479a0-7158-4a49-90d3-bf43bc69a974'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:11.371373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f08e43a5984a9398b3d5ec41e4282c', 'request_id': 'req-d9b479a0-7158-4a49-90d3-bf43bc69a974'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:11.392320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] The token cache is being invalidate because project e0f08e43a5984a9398b3d5ec41e4282c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:11.393075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:11.394094 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d9b479a0-7158-4a49-90d3-bf43bc69a974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:11.396423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1069/2148] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:11 2022] DELETE /identity/v3/projects/e0f08e43a5984a9398b3d5ec41e4282c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:52:11.401606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bedd29-592f-4963-b597-7d474882cf7d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:11.401606 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bedd29-592f-4963-b597-7d474882cf7d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:11.401967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bedd29-592f-4963-b597-7d474882cf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:11.436626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f5bedd29-592f-4963-b597-7d474882cf7d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:11.439670 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f5bedd29-592f-4963-b597-7d474882cf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:11.475932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:11.545551 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1080/2149] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:11.553503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ab9e5-7caa-4f5c-ba23-6128aa3b45fa None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:11.554116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ab9e5-7caa-4f5c-ba23-6128aa3b45fa None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:11.554740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ab9e5-7caa-4f5c-ba23-6128aa3b45fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:11.590394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-612ab9e5-7caa-4f5c-ba23-6128aa3b45fa None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:11.591952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-612ab9e5-7caa-4f5c-ba23-6128aa3b45fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:11.603283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1070/2150] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:11.609187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bedd29-592f-4963-b597-7d474882cf7d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:11.626338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bedd29-592f-4963-b597-7d474882cf7d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:11.631229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d438c97-a98d-4580-a1da-938019389cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:11.631908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d438c97-a98d-4580-a1da-938019389cec admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:11.632256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d438c97-a98d-4580-a1da-938019389cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:11.632595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d438c97-a98d-4580-a1da-938019389cec admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:11.633278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d438c97-a98d-4580-a1da-938019389cec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:52:11.633603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d438c97-a98d-4580-a1da-938019389cec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:11.635316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d438c97-a98d-4580-a1da-938019389cec admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:11.651149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1081/2151] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:52:11.656003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ab9e5-7caa-4f5c-ba23-6128aa3b45fa None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:11.667004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ab9e5-7caa-4f5c-ba23-6128aa3b45fa None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:11.672104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8730898-bfa8-4401-b023-2391322e8b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:11.672724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8730898-bfa8-4401-b023-2391322e8b88 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:11.673005 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8730898-bfa8-4401-b023-2391322e8b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:11.673281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8730898-bfa8-4401-b023-2391322e8b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:11.674086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8730898-bfa8-4401-b023-2391322e8b88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:11.676138 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8730898-bfa8-4401-b023-2391322e8b88 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:11.725285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1071/2152] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:11 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:52:11.726466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d438c97-a98d-4580-a1da-938019389cec admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:11.737961 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d438c97-a98d-4580-a1da-938019389cec admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:11.813292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e2279-5545-4dac-8855-40d7febcd492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:11.814379 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e2279-5545-4dac-8855-40d7febcd492 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:11.814619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e2279-5545-4dac-8855-40d7febcd492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:11.815146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e2279-5545-4dac-8855-40d7febcd492 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:11.816010 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e2279-5545-4dac-8855-40d7febcd492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:11.818277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e2279-5545-4dac-8855-40d7febcd492 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:11.850196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1082/2153] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:11 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:11.853114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8730898-bfa8-4401-b023-2391322e8b88 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:11.864787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8730898-bfa8-4401-b023-2391322e8b88 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:11.869711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7182c5-7e2b-4dbb-a1a5-8c28019ea626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:11.870405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7182c5-7e2b-4dbb-a1a5-8c28019ea626 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:11.870682 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7182c5-7e2b-4dbb-a1a5-8c28019ea626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:11.870961 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7182c5-7e2b-4dbb-a1a5-8c28019ea626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:11.871608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7182c5-7e2b-4dbb-a1a5-8c28019ea626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:11.873881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7182c5-7e2b-4dbb-a1a5-8c28019ea626 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:11.884561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1072/2154] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:11.888849 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e2279-5545-4dac-8855-40d7febcd492 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:11.900323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e2279-5545-4dac-8855-40d7febcd492 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:11.905439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8b48c-edf7-4a3f-9c16-fcaf5d998be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:11.906018 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8b48c-edf7-4a3f-9c16-fcaf5d998be9 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:11.906327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8b48c-edf7-4a3f-9c16-fcaf5d998be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:11.906595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8b48c-edf7-4a3f-9c16-fcaf5d998be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:11.907231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8b48c-edf7-4a3f-9c16-fcaf5d998be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:11.908810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8b48c-edf7-4a3f-9c16-fcaf5d998be9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:11.920386 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1083/2155] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:11.924596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7182c5-7e2b-4dbb-a1a5-8c28019ea626 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:11.935007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7182c5-7e2b-4dbb-a1a5-8c28019ea626 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:11.939503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81da1cf-b002-4582-a0a5-a751db8c9b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:11.940139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81da1cf-b002-4582-a0a5-a751db8c9b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:11.940418 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81da1cf-b002-4582-a0a5-a751db8c9b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:11.940692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81da1cf-b002-4582-a0a5-a751db8c9b79 admin admin] PATH_INFO: `/v3/projects/0a8822df10c6401c8d35d552984330fc/users/c424209448834b2aac0a112c8ef17197/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:11.955701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81da1cf-b002-4582-a0a5-a751db8c9b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8822df10c6401c8d35d552984330fc, user_id=c424209448834b2aac0a112c8ef17197, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:11.958641 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81da1cf-b002-4582-a0a5-a751db8c9b79 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:11.976231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1073/2156] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:11 2022] PUT /identity/v3/projects/0a8822df10c6401c8d35d552984330fc/users/c424209448834b2aac0a112c8ef17197/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:11.988106 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56c315-9a75-4c53-b9aa-66e566d967d3 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:11.988398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56c315-9a75-4c53-b9aa-66e566d967d3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:11.988662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56c315-9a75-4c53-b9aa-66e566d967d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:12.024231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-2b56c315-9a75-4c53-b9aa-66e566d967d3 None None] MFA Rules not processed for user `c424209448834b2aac0a112c8ef17197`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:12.025530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2b56c315-9a75-4c53-b9aa-66e566d967d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:12.094729 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1084/2157] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:12.106842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81da1cf-b002-4582-a0a5-a751db8c9b79 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:12.108417 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b81da1cf-b002-4582-a0a5-a751db8c9b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:12.214471 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81da1cf-b002-4582-a0a5-a751db8c9b79 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:12.220944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd93ffa-a6a6-4fc2-a048-f50835b464fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:12.221602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd93ffa-a6a6-4fc2-a048-f50835b464fe service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:12.221877 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd93ffa-a6a6-4fc2-a048-f50835b464fe service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:12.222148 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd93ffa-a6a6-4fc2-a048-f50835b464fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:12.232404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd93ffa-a6a6-4fc2-a048-f50835b464fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:12.233998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd93ffa-a6a6-4fc2-a048-f50835b464fe service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:12.238690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:12.243812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1074/2158] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:12.537438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56c315-9a75-4c53-b9aa-66e566d967d3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:12.578824 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:12.617758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56c315-9a75-4c53-b9aa-66e566d967d3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:12.623072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:12.623692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:12.623960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:12.624314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] PATH_INFO: `/v3/users/c424209448834b2aac0a112c8ef17197` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:12.625435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c424209448834b2aac0a112c8ef17197)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:12.627432 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:12.680897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c424209448834b2aac0a112c8ef17197'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:12.681638 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c424209448834b2aac0a112c8ef17197'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:12.697056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c424209448834b2aac0a112c8ef17197'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:12.697940 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c424209448834b2aac0a112c8ef17197'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:12.706383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c424209448834b2aac0a112c8ef17197'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:12.720427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1085/2159] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:12 2022] DELETE /identity/v3/users/c424209448834b2aac0a112c8ef17197 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:12.725041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be675-d355-428e-80ef-d1428766c5ef None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:12.725459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be675-d355-428e-80ef-d1428766c5ef None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:12.725807 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be675-d355-428e-80ef-d1428766c5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:12.745357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3d3be675-d355-428e-80ef-d1428766c5ef None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:12.747379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3d3be675-d355-428e-80ef-d1428766c5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:12.812302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1075/2160] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:12.826240 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:12.828762 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:12.889783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8ea54-fcc7-4ec6-8350-c2d2b4f4d5ec admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:12.893351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74890377-fe71-4f52-9817-32ff9284d521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:12.893944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74890377-fe71-4f52-9817-32ff9284d521 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:12.894257 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74890377-fe71-4f52-9817-32ff9284d521 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:12.894525 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74890377-fe71-4f52-9817-32ff9284d521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:12.904527 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74890377-fe71-4f52-9817-32ff9284d521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:12.906220 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74890377-fe71-4f52-9817-32ff9284d521 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:12.916101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1086/2161] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:52:13.249559 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3be675-d355-428e-80ef-d1428766c5ef None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:13.251102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3d3be675-d355-428e-80ef-d1428766c5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:13.271439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3be675-d355-428e-80ef-d1428766c5ef None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:13.277018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:13.278433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:13.279058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:13.279777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] PATH_INFO: `/v3/projects/0a8822df10c6401c8d35d552984330fc` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:13.281779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a8822df10c6401c8d35d552984330fc)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:13.284047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:13.336590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a8822df10c6401c8d35d552984330fc', 'request_id': 'req-84eeac4c-0c7f-4da2-b987-25e2eabf309a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:13.344299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a8822df10c6401c8d35d552984330fc', 'request_id': 'req-84eeac4c-0c7f-4da2-b987-25e2eabf309a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:13.350589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a8822df10c6401c8d35d552984330fc', 'request_id': 'req-84eeac4c-0c7f-4da2-b987-25e2eabf309a'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:13.361116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] The token cache is being invalidate because project 0a8822df10c6401c8d35d552984330fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:13.361398 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:13.362016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-84eeac4c-0c7f-4da2-b987-25e2eabf309a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:13.363599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1076/2162] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:13 2022] DELETE /identity/v3/projects/0a8822df10c6401c8d35d552984330fc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:13.368540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0dd4b-7607-4cf0-bc53-a36b6b15bb51 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:13.368886 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0dd4b-7607-4cf0-bc53-a36b6b15bb51 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:13.369155 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0dd4b-7607-4cf0-bc53-a36b6b15bb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:13.390503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ddb0dd4b-7607-4cf0-bc53-a36b6b15bb51 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:13.391768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ddb0dd4b-7607-4cf0-bc53-a36b6b15bb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:13.439308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:13.456511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1087/2163] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:52:13.467155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027055e8-b379-4036-af57-ccfef7aa60b9 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:13.467442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027055e8-b379-4036-af57-ccfef7aa60b9 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:13.467739 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027055e8-b379-4036-af57-ccfef7aa60b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:13.488495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-027055e8-b379-4036-af57-ccfef7aa60b9 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:13.489925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-027055e8-b379-4036-af57-ccfef7aa60b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:13.500361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1077/2164] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:52:13.505849 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0dd4b-7607-4cf0-bc53-a36b6b15bb51 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:13.517337 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0dd4b-7607-4cf0-bc53-a36b6b15bb51 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:13.522438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a03f92-e33b-4842-a804-ca91deb5a013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:13.523066 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a03f92-e33b-4842-a804-ca91deb5a013 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:13.523346 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a03f92-e33b-4842-a804-ca91deb5a013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:13.523778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a03f92-e33b-4842-a804-ca91deb5a013 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:13.524436 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a03f92-e33b-4842-a804-ca91deb5a013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:52:13.524743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a03f92-e33b-4842-a804-ca91deb5a013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:13.526558 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a03f92-e33b-4842-a804-ca91deb5a013 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:13.545330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1088/2165] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:52:13.549856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027055e8-b379-4036-af57-ccfef7aa60b9 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:13.561984 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027055e8-b379-4036-af57-ccfef7aa60b9 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:13.567492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a612578-d673-44ac-8188-5d3a5779ea0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:13.568320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a612578-d673-44ac-8188-5d3a5779ea0b admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:13.568727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a612578-d673-44ac-8188-5d3a5779ea0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:13.569067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a612578-d673-44ac-8188-5d3a5779ea0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:13.569820 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a612578-d673-44ac-8188-5d3a5779ea0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:13.571822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a612578-d673-44ac-8188-5d3a5779ea0b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:13.604894 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1078/2166] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:13 2022] POST /identity/v3/projects => generated 405 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:13.609235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a03f92-e33b-4842-a804-ca91deb5a013 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:13.622952 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a03f92-e33b-4842-a804-ca91deb5a013 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:13.705491 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f0de3-6305-4ff4-aee4-abddb1195b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:13.706255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f0de3-6305-4ff4-aee4-abddb1195b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:13.706532 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f0de3-6305-4ff4-aee4-abddb1195b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:13.706797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f0de3-6305-4ff4-aee4-abddb1195b03 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:13.707454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f0de3-6305-4ff4-aee4-abddb1195b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:13.709402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f0de3-6305-4ff4-aee4-abddb1195b03 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:13.741429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1089/2167] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:13 2022] POST /identity/v3/users => generated 303 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:13.743686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a612578-d673-44ac-8188-5d3a5779ea0b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:13.757265 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a612578-d673-44ac-8188-5d3a5779ea0b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:13.764564 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c64c8a-0546-4621-be55-1a2aff0eb62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:13.765373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c64c8a-0546-4621-be55-1a2aff0eb62b admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:13.765799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c64c8a-0546-4621-be55-1a2aff0eb62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:13.766267 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c64c8a-0546-4621-be55-1a2aff0eb62b admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:13.767101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c64c8a-0546-4621-be55-1a2aff0eb62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:13.769318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c64c8a-0546-4621-be55-1a2aff0eb62b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:13.786197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1079/2168] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:13.787818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f0de3-6305-4ff4-aee4-abddb1195b03 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:13.798909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f0de3-6305-4ff4-aee4-abddb1195b03 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:13.804192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56a82c-edff-4f62-847f-67961d954c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:13.804810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56a82c-edff-4f62-847f-67961d954c82 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:13.805091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56a82c-edff-4f62-847f-67961d954c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:13.805364 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56a82c-edff-4f62-847f-67961d954c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:13.806014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56a82c-edff-4f62-847f-67961d954c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:13.807669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56a82c-edff-4f62-847f-67961d954c82 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:13.823099 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1090/2169] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:13.824079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c64c8a-0546-4621-be55-1a2aff0eb62b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:13.834765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c64c8a-0546-4621-be55-1a2aff0eb62b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:13.839353 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6fe17-1176-47b2-a7a6-048e1533e023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:13.840015 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6fe17-1176-47b2-a7a6-048e1533e023 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:13.840315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6fe17-1176-47b2-a7a6-048e1533e023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:13.840606 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6fe17-1176-47b2-a7a6-048e1533e023 admin admin] PATH_INFO: `/v3/projects/a7189cbfdd664fc9b482d52eb02ac6de/users/afd38bd816ee4ee3ab8e68c7983a81bd/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:13.854807 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6fe17-1176-47b2-a7a6-048e1533e023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7189cbfdd664fc9b482d52eb02ac6de, user_id=afd38bd816ee4ee3ab8e68c7983a81bd, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:13.857473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6fe17-1176-47b2-a7a6-048e1533e023 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:13.877518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af2fcc-39af-4247-a66e-343efc05bec0 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:13.878114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af2fcc-39af-4247-a66e-343efc05bec0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:13.878114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af2fcc-39af-4247-a66e-343efc05bec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:13.880241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1080/2170] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:13 2022] PUT /identity/v3/projects/a7189cbfdd664fc9b482d52eb02ac6de/users/afd38bd816ee4ee3ab8e68c7983a81bd/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:52:13.899120 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-e4af2fcc-39af-4247-a66e-343efc05bec0 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:13.900827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e4af2fcc-39af-4247-a66e-343efc05bec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:13.961156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1091/2171] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:13.982394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6fe17-1176-47b2-a7a6-048e1533e023 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:13.984396 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1dd6fe17-1176-47b2-a7a6-048e1533e023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:14.010551 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:14.055989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6fe17-1176-47b2-a7a6-048e1533e023 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:14.060478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac98c8-0c7e-496f-ae20-bcbba7b659c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:14.061136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac98c8-0c7e-496f-ae20-bcbba7b659c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:14.061493 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac98c8-0c7e-496f-ae20-bcbba7b659c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:14.061770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac98c8-0c7e-496f-ae20-bcbba7b659c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:14.071632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac98c8-0c7e-496f-ae20-bcbba7b659c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:14.073073 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac98c8-0c7e-496f-ae20-bcbba7b659c5 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:14.084012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1081/2172] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:14.885022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539d1fe-b36b-41f4-8ba9-c4e1ec202bd4 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:14.885598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539d1fe-b36b-41f4-8ba9-c4e1ec202bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:14.886154 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539d1fe-b36b-41f4-8ba9-c4e1ec202bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:14.939117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-2539d1fe-b36b-41f4-8ba9-c4e1ec202bd4 None None] MFA Rules not processed for user `afd38bd816ee4ee3ab8e68c7983a81bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:14.940954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2539d1fe-b36b-41f4-8ba9-c4e1ec202bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:15.021548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:15.051120 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1092/2173] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:14 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:15.055831 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac98c8-0c7e-496f-ae20-bcbba7b659c5 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:15.066361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac98c8-0c7e-496f-ae20-bcbba7b659c5 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:15.072788 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec97b4-7dd7-4adb-822c-030638533f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:15.073500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec97b4-7dd7-4adb-822c-030638533f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:15.073844 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec97b4-7dd7-4adb-822c-030638533f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:15.074201 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec97b4-7dd7-4adb-822c-030638533f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:15.075455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec97b4-7dd7-4adb-822c-030638533f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:15.077428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec97b4-7dd7-4adb-822c-030638533f9d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:15.113116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1082/2174] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:15 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:15.117752 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539d1fe-b36b-41f4-8ba9-c4e1ec202bd4 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:15.129842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539d1fe-b36b-41f4-8ba9-c4e1ec202bd4 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:15.209398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda5944-da89-482e-ba4f-03b1b591cd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:15.209398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda5944-da89-482e-ba4f-03b1b591cd21 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:15.209398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda5944-da89-482e-ba4f-03b1b591cd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:15.209398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda5944-da89-482e-ba4f-03b1b591cd21 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:15.210134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda5944-da89-482e-ba4f-03b1b591cd21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:15.214693 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda5944-da89-482e-ba4f-03b1b591cd21 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:15.240277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1093/2175] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:15 2022] POST /identity/v3/users => generated 309 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:15.244471 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec97b4-7dd7-4adb-822c-030638533f9d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:15.258097 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec97b4-7dd7-4adb-822c-030638533f9d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:15.264627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbb2dd-7772-4c6b-9e64-f0ea6d37dc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:15.265245 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbb2dd-7772-4c6b-9e64-f0ea6d37dc41 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:15.265528 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbb2dd-7772-4c6b-9e64-f0ea6d37dc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:15.265896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbb2dd-7772-4c6b-9e64-f0ea6d37dc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:15.266595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbb2dd-7772-4c6b-9e64-f0ea6d37dc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:15.268222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbb2dd-7772-4c6b-9e64-f0ea6d37dc41 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:15.281759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1083/2176] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:52:15.286193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda5944-da89-482e-ba4f-03b1b591cd21 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:15.296906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda5944-da89-482e-ba4f-03b1b591cd21 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:15.301776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a8b49-bb0b-47e5-a696-e5da1150d8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:15.302402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a8b49-bb0b-47e5-a696-e5da1150d8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:15.302678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a8b49-bb0b-47e5-a696-e5da1150d8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:15.302947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a8b49-bb0b-47e5-a696-e5da1150d8c9 admin admin] PATH_INFO: `/v3/projects/814e50dab927479dae0a6be1aa99b438/users/65ddd15c52014b14829185ef5e4ad624/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:15.319137 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a8b49-bb0b-47e5-a696-e5da1150d8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=814e50dab927479dae0a6be1aa99b438, user_id=65ddd15c52014b14829185ef5e4ad624, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:15.321800 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a8b49-bb0b-47e5-a696-e5da1150d8c9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:15.337756 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1094/2177] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:15 2022] PUT /identity/v3/projects/814e50dab927479dae0a6be1aa99b438/users/65ddd15c52014b14829185ef5e4ad624/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:15.343041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbb2dd-7772-4c6b-9e64-f0ea6d37dc41 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:15.354381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbb2dd-7772-4c6b-9e64-f0ea6d37dc41 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:15.422858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c104161-306d-4565-a6e9-ab61cf2aa99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:15.423907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c104161-306d-4565-a6e9-ab61cf2aa99f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:15.424242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c104161-306d-4565-a6e9-ab61cf2aa99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:15.424511 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c104161-306d-4565-a6e9-ab61cf2aa99f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:15.425151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c104161-306d-4565-a6e9-ab61cf2aa99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:15.426873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c104161-306d-4565-a6e9-ab61cf2aa99f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:15.438929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1084/2178] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:52:15.443316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a8b49-bb0b-47e5-a696-e5da1150d8c9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:15.456337 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a8b49-bb0b-47e5-a696-e5da1150d8c9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:15.467034 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f31306-e461-47f0-88a6-7909f98176e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:15.467684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f31306-e461-47f0-88a6-7909f98176e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:15.467957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f31306-e461-47f0-88a6-7909f98176e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:15.468231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f31306-e461-47f0-88a6-7909f98176e1 admin admin] PATH_INFO: `/v3/projects/814e50dab927479dae0a6be1aa99b438/users/65ddd15c52014b14829185ef5e4ad624/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:15.470412 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f31306-e461-47f0-88a6-7909f98176e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=814e50dab927479dae0a6be1aa99b438, user_id=65ddd15c52014b14829185ef5e4ad624, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:15.473696 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f31306-e461-47f0-88a6-7909f98176e1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:15.488901 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1095/2179] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:15 2022] PUT /identity/v3/projects/814e50dab927479dae0a6be1aa99b438/users/65ddd15c52014b14829185ef5e4ad624/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:16.430254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3aeef8-127d-49ea-bf4f-b18ed0d3ee9c None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:16.430785 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3aeef8-127d-49ea-bf4f-b18ed0d3ee9c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:16.430878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3aeef8-127d-49ea-bf4f-b18ed0d3ee9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:16.470966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6f3aeef8-127d-49ea-bf4f-b18ed0d3ee9c None None] MFA Rules not processed for user `65ddd15c52014b14829185ef5e4ad624`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:16.472120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6f3aeef8-127d-49ea-bf4f-b18ed0d3ee9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:16.559272 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1085/2180] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:16 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:17.781413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f31306-e461-47f0-88a6-7909f98176e1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:17.849016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f31306-e461-47f0-88a6-7909f98176e1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:17.853731 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:17.854400 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:17.854681 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:17.854952 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] PATH_INFO: `/v3/users/afd38bd816ee4ee3ab8e68c7983a81bd` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:17.856094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd38bd816ee4ee3ab8e68c7983a81bd)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:17.858316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:17.911636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:17.918897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd38bd816ee4ee3ab8e68c7983a81bd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:17.920181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd38bd816ee4ee3ab8e68c7983a81bd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:17.938297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd38bd816ee4ee3ab8e68c7983a81bd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:17.939069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd38bd816ee4ee3ab8e68c7983a81bd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:17.947582 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd38bd816ee4ee3ab8e68c7983a81bd'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:17.964657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1096/2181] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:17 2022] DELETE /identity/v3/users/afd38bd816ee4ee3ab8e68c7983a81bd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:18.160348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3aeef8-127d-49ea-bf4f-b18ed0d3ee9c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:18.173926 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3aeef8-127d-49ea-bf4f-b18ed0d3ee9c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:18.185141 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:18.261019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:18.261652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:18.261932 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:18.262815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] PATH_INFO: `/v3/projects/a7189cbfdd664fc9b482d52eb02ac6de` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:18.263989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7189cbfdd664fc9b482d52eb02ac6de)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:18.265938 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:18.324697 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7189cbfdd664fc9b482d52eb02ac6de', 'request_id': 'req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:18.333997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7189cbfdd664fc9b482d52eb02ac6de', 'request_id': 'req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:18.340354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7189cbfdd664fc9b482d52eb02ac6de', 'request_id': 'req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:18.351014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] The token cache is being invalidate because project a7189cbfdd664fc9b482d52eb02ac6de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:18.351302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:18.351947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:18.353439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1086/2182] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:18 2022] DELETE /identity/v3/projects/a7189cbfdd664fc9b482d52eb02ac6de => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:18.357605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:18.359106 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:18.375659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efbcdf-c75e-489a-bf7b-b6c127de00bd admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:18.452984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47594780-dd6f-4c98-bd91-7e4e05b38ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:18.453618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47594780-dd6f-4c98-bd91-7e4e05b38ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:18.453895 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47594780-dd6f-4c98-bd91-7e4e05b38ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:18.454184 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47594780-dd6f-4c98-bd91-7e4e05b38ecb admin admin] PATH_INFO: `/v3/users/65ddd15c52014b14829185ef5e4ad624` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:18.455348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47594780-dd6f-4c98-bd91-7e4e05b38ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ddd15c52014b14829185ef5e4ad624)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:18.457289 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47594780-dd6f-4c98-bd91-7e4e05b38ecb admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:18.526539 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-47594780-dd6f-4c98-bd91-7e4e05b38ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ddd15c52014b14829185ef5e4ad624'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:18.527196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-47594780-dd6f-4c98-bd91-7e4e05b38ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ddd15c52014b14829185ef5e4ad624'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:18.539769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-47594780-dd6f-4c98-bd91-7e4e05b38ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ddd15c52014b14829185ef5e4ad624'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:18.540429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-47594780-dd6f-4c98-bd91-7e4e05b38ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ddd15c52014b14829185ef5e4ad624'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:18.547609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-47594780-dd6f-4c98-bd91-7e4e05b38ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ddd15c52014b14829185ef5e4ad624'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:18.577175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1097/2183] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:18 2022] DELETE /identity/v3/users/65ddd15c52014b14829185ef5e4ad624 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:18.853448 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:18.854912 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:18.893101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5525dd-1407-4ea4-bd5f-c22a6db6be34 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:18.979392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:18.980578 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:18.982109 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:18.982730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] PATH_INFO: `/v3/projects/814e50dab927479dae0a6be1aa99b438` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:18.984368 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=814e50dab927479dae0a6be1aa99b438)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:18.986835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:19.045840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:19.050715 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '814e50dab927479dae0a6be1aa99b438', 'request_id': 'req-cbbfe208-69a8-455b-94be-4e79ccdc5549'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:19.066183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '814e50dab927479dae0a6be1aa99b438', 'request_id': 'req-cbbfe208-69a8-455b-94be-4e79ccdc5549'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:19.071348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '814e50dab927479dae0a6be1aa99b438', 'request_id': 'req-cbbfe208-69a8-455b-94be-4e79ccdc5549'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:19.084379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] The token cache is being invalidate because project 814e50dab927479dae0a6be1aa99b438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:19.084939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:19.085679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-cbbfe208-69a8-455b-94be-4e79ccdc5549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:19.092781 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b74ae-1f81-4ce5-b0de-1ef79e8fdd7f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:19.093313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b74ae-1f81-4ce5-b0de-1ef79e8fdd7f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:19.093313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b74ae-1f81-4ce5-b0de-1ef79e8fdd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:19.096122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1087/2184] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:18 2022] DELETE /identity/v3/projects/814e50dab927479dae0a6be1aa99b438 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:19.124423 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b93b74ae-1f81-4ce5-b0de-1ef79e8fdd7f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:19.126076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b93b74ae-1f81-4ce5-b0de-1ef79e8fdd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:19.233736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:19.271313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1098/2185] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:19.281692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a5007-9995-4286-8504-861581790aae None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:19.282335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a5007-9995-4286-8504-861581790aae None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:19.282575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a5007-9995-4286-8504-861581790aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:19.310425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-e47a5007-9995-4286-8504-861581790aae None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:19.311925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e47a5007-9995-4286-8504-861581790aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:19.322592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1088/2186] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:52:19.327270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b74ae-1f81-4ce5-b0de-1ef79e8fdd7f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:19.338947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b74ae-1f81-4ce5-b0de-1ef79e8fdd7f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:19.344958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ca273-e0f7-4fa5-a73f-967471fe204d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:19.345854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca273-e0f7-4fa5-a73f-967471fe204d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:19.346362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca273-e0f7-4fa5-a73f-967471fe204d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:19.346655 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ca273-e0f7-4fa5-a73f-967471fe204d admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:19.347547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ca273-e0f7-4fa5-a73f-967471fe204d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:52:19.347922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ca273-e0f7-4fa5-a73f-967471fe204d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:19.349524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ca273-e0f7-4fa5-a73f-967471fe204d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:19.361708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1099/2187] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:52:19.366223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a5007-9995-4286-8504-861581790aae None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:19.375428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a5007-9995-4286-8504-861581790aae None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:19.380371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9aa7ea-2640-4c82-878f-c07a939bcb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:19.380977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9aa7ea-2640-4c82-878f-c07a939bcb1b admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:19.381258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9aa7ea-2640-4c82-878f-c07a939bcb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:19.381531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9aa7ea-2640-4c82-878f-c07a939bcb1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:19.382207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9aa7ea-2640-4c82-878f-c07a939bcb1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:19.384191 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9aa7ea-2640-4c82-878f-c07a939bcb1b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:19.414046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1089/2188] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:19 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:19.418036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ca273-e0f7-4fa5-a73f-967471fe204d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:19.432032 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ca273-e0f7-4fa5-a73f-967471fe204d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:19.492563 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14824693-0b89-4e99-b756-33bbdf6da0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:19.493171 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824693-0b89-4e99-b756-33bbdf6da0fc admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:19.493453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824693-0b89-4e99-b756-33bbdf6da0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:19.493727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824693-0b89-4e99-b756-33bbdf6da0fc admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:19.494378 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14824693-0b89-4e99-b756-33bbdf6da0fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:19.496290 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14824693-0b89-4e99-b756-33bbdf6da0fc admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:19.518073 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1100/2189] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:19 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:52:19.522066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9aa7ea-2640-4c82-878f-c07a939bcb1b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:19.533488 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9aa7ea-2640-4c82-878f-c07a939bcb1b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:19.538197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7fe6d-7a68-4e22-9d70-85a10a237d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:19.538805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7fe6d-7a68-4e22-9d70-85a10a237d76 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:19.539081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7fe6d-7a68-4e22-9d70-85a10a237d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:19.539370 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7fe6d-7a68-4e22-9d70-85a10a237d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:19.539994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7fe6d-7a68-4e22-9d70-85a10a237d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:19.541584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7fe6d-7a68-4e22-9d70-85a10a237d76 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:19.551770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1090/2190] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:19.555768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14824693-0b89-4e99-b756-33bbdf6da0fc admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:19.566036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14824693-0b89-4e99-b756-33bbdf6da0fc admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:19.570980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e66bad-45e7-494c-a565-520458604be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:19.571632 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e66bad-45e7-494c-a565-520458604be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:19.571908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e66bad-45e7-494c-a565-520458604be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:19.572179 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e66bad-45e7-494c-a565-520458604be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:19.572854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e66bad-45e7-494c-a565-520458604be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:19.575212 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e66bad-45e7-494c-a565-520458604be3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:19.589720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1101/2191] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:19.592075 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7fe6d-7a68-4e22-9d70-85a10a237d76 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:19.602849 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7fe6d-7a68-4e22-9d70-85a10a237d76 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:19.608541 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4b50b-eb19-4a01-9a86-68e02a869ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:19.609464 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4b50b-eb19-4a01-9a86-68e02a869ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:19.609964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4b50b-eb19-4a01-9a86-68e02a869ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:19.610537 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4b50b-eb19-4a01-9a86-68e02a869ac9 admin admin] PATH_INFO: `/v3/projects/a96372fe29414bd59531a36979ab3375/users/da404cca2b70454082caab063026405b/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:19.626436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4b50b-eb19-4a01-9a86-68e02a869ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96372fe29414bd59531a36979ab3375, user_id=da404cca2b70454082caab063026405b, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:19.629091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4b50b-eb19-4a01-9a86-68e02a869ac9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:19.643448 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1091/2192] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:19 2022] PUT /identity/v3/projects/a96372fe29414bd59531a36979ab3375/users/da404cca2b70454082caab063026405b/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:19.654730 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba4069-da5a-4b22-8442-2b141fe7957d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:19.654947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba4069-da5a-4b22-8442-2b141fe7957d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:19.655475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba4069-da5a-4b22-8442-2b141fe7957d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:19.686142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c9ba4069-da5a-4b22-8442-2b141fe7957d None None] MFA Rules not processed for user `da404cca2b70454082caab063026405b`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:19.687788 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c9ba4069-da5a-4b22-8442-2b141fe7957d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:19.774574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1102/2193] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:19.779514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4b50b-eb19-4a01-9a86-68e02a869ac9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:19.862941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4b50b-eb19-4a01-9a86-68e02a869ac9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:19.879944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d210f4-6706-4537-a724-e9a08a9762da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:19.880721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d210f4-6706-4537-a724-e9a08a9762da admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:19.881005 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d210f4-6706-4537-a724-e9a08a9762da admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:19.881282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d210f4-6706-4537-a724-e9a08a9762da admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:19.881944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d210f4-6706-4537-a724-e9a08a9762da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:19.883880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d210f4-6706-4537-a724-e9a08a9762da admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:19.912991 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1092/2194] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:19 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:19.916869 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba4069-da5a-4b22-8442-2b141fe7957d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:19.931316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba4069-da5a-4b22-8442-2b141fe7957d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:19.993036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:20.041132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a92907-8cc4-4517-93ac-088a5aab8d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:20.041750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a92907-8cc4-4517-93ac-088a5aab8d93 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:20.042026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a92907-8cc4-4517-93ac-088a5aab8d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:20.042335 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a92907-8cc4-4517-93ac-088a5aab8d93 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:20.042972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a92907-8cc4-4517-93ac-088a5aab8d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:20.044937 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a92907-8cc4-4517-93ac-088a5aab8d93 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:20.068689 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1103/2195] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:19 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:52:20.072915 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d210f4-6706-4537-a724-e9a08a9762da admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:20.086873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d210f4-6706-4537-a724-e9a08a9762da admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:20.092310 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d7ba0-1020-41e2-9e70-dd4ad9d9b80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:20.092998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d7ba0-1020-41e2-9e70-dd4ad9d9b80b admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:20.093351 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d7ba0-1020-41e2-9e70-dd4ad9d9b80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:20.093638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d7ba0-1020-41e2-9e70-dd4ad9d9b80b admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:20.094312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d7ba0-1020-41e2-9e70-dd4ad9d9b80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:20.095968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d7ba0-1020-41e2-9e70-dd4ad9d9b80b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:20.107907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1093/2196] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:52:20.112021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a92907-8cc4-4517-93ac-088a5aab8d93 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:20.123130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a92907-8cc4-4517-93ac-088a5aab8d93 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:20.127725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fa612-b24f-4449-a460-c73f77da2b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:20.128369 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa612-b24f-4449-a460-c73f77da2b23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:20.128642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa612-b24f-4449-a460-c73f77da2b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:20.128907 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa612-b24f-4449-a460-c73f77da2b23 admin admin] PATH_INFO: `/v3/projects/769ce849c6af49cea5fd8e3cc4bda8df/users/96cfc9ede0ce4bcf8a08aa5ad57237b3/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:20.143596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fa612-b24f-4449-a460-c73f77da2b23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=769ce849c6af49cea5fd8e3cc4bda8df, user_id=96cfc9ede0ce4bcf8a08aa5ad57237b3, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:20.146573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fa612-b24f-4449-a460-c73f77da2b23 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:20.165660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1104/2197] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:20 2022] PUT /identity/v3/projects/769ce849c6af49cea5fd8e3cc4bda8df/users/96cfc9ede0ce4bcf8a08aa5ad57237b3/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:20.170333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d7ba0-1020-41e2-9e70-dd4ad9d9b80b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:20.182050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d7ba0-1020-41e2-9e70-dd4ad9d9b80b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:20.227701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:20.249463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef3350-edf6-4a0c-af48-512db2a2c25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:20.250069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef3350-edf6-4a0c-af48-512db2a2c25b admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:20.250421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef3350-edf6-4a0c-af48-512db2a2c25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:20.250821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef3350-edf6-4a0c-af48-512db2a2c25b admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:20.251673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef3350-edf6-4a0c-af48-512db2a2c25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:20.253585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef3350-edf6-4a0c-af48-512db2a2c25b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:20.274595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1094/2198] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:20.276479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fa612-b24f-4449-a460-c73f77da2b23 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:20.289309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fa612-b24f-4449-a460-c73f77da2b23 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:20.294597 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5d29d-9624-45ef-8968-896e64688ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:20.295262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5d29d-9624-45ef-8968-896e64688ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:20.295539 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5d29d-9624-45ef-8968-896e64688ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:20.295808 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5d29d-9624-45ef-8968-896e64688ae4 admin admin] PATH_INFO: `/v3/projects/769ce849c6af49cea5fd8e3cc4bda8df/users/96cfc9ede0ce4bcf8a08aa5ad57237b3/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:20.297724 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5d29d-9624-45ef-8968-896e64688ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=769ce849c6af49cea5fd8e3cc4bda8df, user_id=96cfc9ede0ce4bcf8a08aa5ad57237b3, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:20.300389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5d29d-9624-45ef-8968-896e64688ae4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:20.315151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1105/2199] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:20 2022] PUT /identity/v3/projects/769ce849c6af49cea5fd8e3cc4bda8df/users/96cfc9ede0ce4bcf8a08aa5ad57237b3/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:20.326074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547832e8-5e52-4139-abad-6c48c0d45539 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:20.326589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547832e8-5e52-4139-abad-6c48c0d45539 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:20.326697 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547832e8-5e52-4139-abad-6c48c0d45539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:20.357277 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-547832e8-5e52-4139-abad-6c48c0d45539 None None] MFA Rules not processed for user `96cfc9ede0ce4bcf8a08aa5ad57237b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:20.358527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-547832e8-5e52-4139-abad-6c48c0d45539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:20.449890 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1095/2200] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:20.461962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5d29d-9624-45ef-8968-896e64688ae4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:20.463741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-61b5d29d-9624-45ef-8968-896e64688ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:20.523180 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5d29d-9624-45ef-8968-896e64688ae4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:20.526837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5f05a-9a97-438b-bb53-5d5ee49b77c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:20.527437 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f05a-9a97-438b-bb53-5d5ee49b77c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:20.528245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f05a-9a97-438b-bb53-5d5ee49b77c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:20.528518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5f05a-9a97-438b-bb53-5d5ee49b77c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:20.537616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5f05a-9a97-438b-bb53-5d5ee49b77c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:20.539058 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5f05a-9a97-438b-bb53-5d5ee49b77c7 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:20.549444 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1106/2201] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:20.940062 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547832e8-5e52-4139-abad-6c48c0d45539 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:21.002247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547832e8-5e52-4139-abad-6c48c0d45539 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:21.006778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:21.007405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:21.007688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:21.007963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] PATH_INFO: `/v3/users/da404cca2b70454082caab063026405b` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:21.009073 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da404cca2b70454082caab063026405b)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:21.011198 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:21.059187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da404cca2b70454082caab063026405b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.060049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da404cca2b70454082caab063026405b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.060645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da404cca2b70454082caab063026405b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.067826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da404cca2b70454082caab063026405b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.075451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:21.081516 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da404cca2b70454082caab063026405b'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.096355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1096/2202] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:20 2022] DELETE /identity/v3/users/da404cca2b70454082caab063026405b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:21.100868 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed824f7-9b4e-4a98-a4a2-07aad284d031 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:21.101175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed824f7-9b4e-4a98-a4a2-07aad284d031 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:21.101470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed824f7-9b4e-4a98-a4a2-07aad284d031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:21.120769 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9ed824f7-9b4e-4a98-a4a2-07aad284d031 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:21.121967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9ed824f7-9b4e-4a98-a4a2-07aad284d031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:21.183724 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1107/2203] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:21.194430 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:21.195862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:21.259730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9949cce-7485-432e-b2eb-6c5b3f0a1e88 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:21.264086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971af54-35ed-43e6-9836-cc8197dd5482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:21.264693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971af54-35ed-43e6-9836-cc8197dd5482 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:21.264970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971af54-35ed-43e6-9836-cc8197dd5482 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:21.265235 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971af54-35ed-43e6-9836-cc8197dd5482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:21.274114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971af54-35ed-43e6-9836-cc8197dd5482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:21.275556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971af54-35ed-43e6-9836-cc8197dd5482 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:21.285077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1097/2204] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:21.557771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed824f7-9b4e-4a98-a4a2-07aad284d031 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:21.559944 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9ed824f7-9b4e-4a98-a4a2-07aad284d031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:21.572373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed824f7-9b4e-4a98-a4a2-07aad284d031 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:21.578241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:21.578880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:21.579163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:21.579489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] PATH_INFO: `/v3/projects/a96372fe29414bd59531a36979ab3375` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:21.580590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a96372fe29414bd59531a36979ab3375)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:21.582624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:21.606305 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:21.630629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a96372fe29414bd59531a36979ab3375', 'request_id': 'req-982a5a44-09c5-4a0e-a951-42b14b5499b0'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.636039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a96372fe29414bd59531a36979ab3375', 'request_id': 'req-982a5a44-09c5-4a0e-a951-42b14b5499b0'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.643437 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a96372fe29414bd59531a36979ab3375', 'request_id': 'req-982a5a44-09c5-4a0e-a951-42b14b5499b0'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.654503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] The token cache is being invalidate because project a96372fe29414bd59531a36979ab3375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:21.654721 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.655614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.657088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1108/2205] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:21 2022] DELETE /identity/v3/projects/a96372fe29414bd59531a36979ab3375 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:21.661205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971af54-35ed-43e6-9836-cc8197dd5482 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:21.671270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971af54-35ed-43e6-9836-cc8197dd5482 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:21.725384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609cffb-462a-4605-90bd-57de3729479f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:21.726008 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609cffb-462a-4605-90bd-57de3729479f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:21.726321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609cffb-462a-4605-90bd-57de3729479f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:21.726595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609cffb-462a-4605-90bd-57de3729479f admin admin] PATH_INFO: `/v3/users/96cfc9ede0ce4bcf8a08aa5ad57237b3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:21.727697 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609cffb-462a-4605-90bd-57de3729479f admin admin] RBAC: Authorizing `identity:delete_user(user_id=96cfc9ede0ce4bcf8a08aa5ad57237b3)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:21.729589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609cffb-462a-4605-90bd-57de3729479f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:21.773986 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f609cffb-462a-4605-90bd-57de3729479f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96cfc9ede0ce4bcf8a08aa5ad57237b3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.774663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f609cffb-462a-4605-90bd-57de3729479f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96cfc9ede0ce4bcf8a08aa5ad57237b3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.775263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f609cffb-462a-4605-90bd-57de3729479f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96cfc9ede0ce4bcf8a08aa5ad57237b3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.782351 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f609cffb-462a-4605-90bd-57de3729479f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96cfc9ede0ce4bcf8a08aa5ad57237b3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.794766 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f609cffb-462a-4605-90bd-57de3729479f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96cfc9ede0ce4bcf8a08aa5ad57237b3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:21.807251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1098/2206] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:21 2022] DELETE /identity/v3/users/96cfc9ede0ce4bcf8a08aa5ad57237b3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:52:22.070827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:22.072404 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:22.087173 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a5a44-09c5-4a0e-a951-42b14b5499b0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:22.144435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:22.145114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.145333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.145605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] PATH_INFO: `/v3/projects/769ce849c6af49cea5fd8e3cc4bda8df` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.146735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] RBAC: Authorizing `identity:delete_project(project_id=769ce849c6af49cea5fd8e3cc4bda8df)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:22.148627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:22.195304 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '769ce849c6af49cea5fd8e3cc4bda8df', 'request_id': 'req-3f98d896-148f-4444-9d95-00ef7495895c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:22.201241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '769ce849c6af49cea5fd8e3cc4bda8df', 'request_id': 'req-3f98d896-148f-4444-9d95-00ef7495895c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:22.208627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '769ce849c6af49cea5fd8e3cc4bda8df', 'request_id': 'req-3f98d896-148f-4444-9d95-00ef7495895c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:22.220236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] The token cache is being invalidate because project 769ce849c6af49cea5fd8e3cc4bda8df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:22.220524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:22.221152 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3f98d896-148f-4444-9d95-00ef7495895c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:22.222901 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1109/2207] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:22 2022] DELETE /identity/v3/projects/769ce849c6af49cea5fd8e3cc4bda8df => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:22.227654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb17d37-85a4-4e25-8972-6ca89ded5a98 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.227940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb17d37-85a4-4e25-8972-6ca89ded5a98 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.228210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb17d37-85a4-4e25-8972-6ca89ded5a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.237873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:22.250417 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2eb17d37-85a4-4e25-8972-6ca89ded5a98 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:22.252432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2eb17d37-85a4-4e25-8972-6ca89ded5a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:22.314383 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1099/2208] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:22.322264 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c791c-f8ce-4a8a-9b1d-6559aa387cd5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.322634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c791c-f8ce-4a8a-9b1d-6559aa387cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.322819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c791c-f8ce-4a8a-9b1d-6559aa387cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.343612 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-191c791c-f8ce-4a8a-9b1d-6559aa387cd5 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:22.344814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-191c791c-f8ce-4a8a-9b1d-6559aa387cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:22.352772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1110/2209] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:22.357233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb17d37-85a4-4e25-8972-6ca89ded5a98 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:22.368495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb17d37-85a4-4e25-8972-6ca89ded5a98 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:22.373388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9c5ff-d1e1-4116-ba00-646b1175d20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:22.373996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9c5ff-d1e1-4116-ba00-646b1175d20c admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.374308 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9c5ff-d1e1-4116-ba00-646b1175d20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.374583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9c5ff-d1e1-4116-ba00-646b1175d20c admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.375397 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9c5ff-d1e1-4116-ba00-646b1175d20c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:52:22.375659 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9c5ff-d1e1-4116-ba00-646b1175d20c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:22.377254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9c5ff-d1e1-4116-ba00-646b1175d20c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:22.392349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1100/2210] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:52:22.396519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c791c-f8ce-4a8a-9b1d-6559aa387cd5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:22.407041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c791c-f8ce-4a8a-9b1d-6559aa387cd5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:22.411678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30d91d-8f29-4463-a904-d5ac120e3c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:22.412269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30d91d-8f29-4463-a904-d5ac120e3c6f admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.412548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30d91d-8f29-4463-a904-d5ac120e3c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.412822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30d91d-8f29-4463-a904-d5ac120e3c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.413457 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30d91d-8f29-4463-a904-d5ac120e3c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:22.415390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30d91d-8f29-4463-a904-d5ac120e3c6f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:22.442968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1111/2211] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:22 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:52:22.447003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9c5ff-d1e1-4116-ba00-646b1175d20c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:22.457783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9c5ff-d1e1-4116-ba00-646b1175d20c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:22.512724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f0c89-c3c8-4033-b7dd-fc2666f56301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:22.513314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f0c89-c3c8-4033-b7dd-fc2666f56301 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.513592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f0c89-c3c8-4033-b7dd-fc2666f56301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.513860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f0c89-c3c8-4033-b7dd-fc2666f56301 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.514521 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f0c89-c3c8-4033-b7dd-fc2666f56301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:22.516432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f0c89-c3c8-4033-b7dd-fc2666f56301 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:22.538303 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1101/2212] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:22 2022] POST /identity/v3/users => generated 281 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:22.542421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30d91d-8f29-4463-a904-d5ac120e3c6f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:22.552610 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30d91d-8f29-4463-a904-d5ac120e3c6f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:22.557332 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fce00-41e7-45d7-ad6b-82d4d2129602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:22.557922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fce00-41e7-45d7-ad6b-82d4d2129602 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.558228 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fce00-41e7-45d7-ad6b-82d4d2129602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.558497 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fce00-41e7-45d7-ad6b-82d4d2129602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.559253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fce00-41e7-45d7-ad6b-82d4d2129602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:22.560820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fce00-41e7-45d7-ad6b-82d4d2129602 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:22.566283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:22.572086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1112/2213] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:52:22.576363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f0c89-c3c8-4033-b7dd-fc2666f56301 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:22.586279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f0c89-c3c8-4033-b7dd-fc2666f56301 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:22.591601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26900e5-f2da-4bab-aff8-c8710d7b55e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:22.592194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26900e5-f2da-4bab-aff8-c8710d7b55e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.592472 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26900e5-f2da-4bab-aff8-c8710d7b55e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.592752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26900e5-f2da-4bab-aff8-c8710d7b55e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.593386 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26900e5-f2da-4bab-aff8-c8710d7b55e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:22.595015 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26900e5-f2da-4bab-aff8-c8710d7b55e2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:22.605675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1102/2214] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:22.610643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fce00-41e7-45d7-ad6b-82d4d2129602 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:22.620464 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fce00-41e7-45d7-ad6b-82d4d2129602 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:22.625121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85763f6-1df7-4519-9039-ba7a48c4f984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:22.625759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85763f6-1df7-4519-9039-ba7a48c4f984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.626038 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85763f6-1df7-4519-9039-ba7a48c4f984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.626344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85763f6-1df7-4519-9039-ba7a48c4f984 admin admin] PATH_INFO: `/v3/projects/66f8abe48c7444689eacae538b5468d2/users/65db05a7965945388af6dca10d9aa12f/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.641080 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85763f6-1df7-4519-9039-ba7a48c4f984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f8abe48c7444689eacae538b5468d2, user_id=65db05a7965945388af6dca10d9aa12f, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:22.643762 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85763f6-1df7-4519-9039-ba7a48c4f984 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:22.658626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1113/2215] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:22 2022] PUT /identity/v3/projects/66f8abe48c7444689eacae538b5468d2/users/65db05a7965945388af6dca10d9aa12f/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:52:22.669020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f5fc8-4fd4-4ae6-9ca7-938c6f60f24e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.669499 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f5fc8-4fd4-4ae6-9ca7-938c6f60f24e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.669594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f5fc8-4fd4-4ae6-9ca7-938c6f60f24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.697223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-ff0f5fc8-4fd4-4ae6-9ca7-938c6f60f24e None None] MFA Rules not processed for user `65db05a7965945388af6dca10d9aa12f`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:22.698488 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ff0f5fc8-4fd4-4ae6-9ca7-938c6f60f24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:22.760271 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:22.765222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1103/2216] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:22.770086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85763f6-1df7-4519-9039-ba7a48c4f984 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:22.831834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85763f6-1df7-4519-9039-ba7a48c4f984 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:22.836384 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf916b-aad9-4496-98d4-f2d6f080a73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:22.836983 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf916b-aad9-4496-98d4-f2d6f080a73a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.837278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf916b-aad9-4496-98d4-f2d6f080a73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.837525 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf916b-aad9-4496-98d4-f2d6f080a73a admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.838149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf916b-aad9-4496-98d4-f2d6f080a73a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:22.840044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf916b-aad9-4496-98d4-f2d6f080a73a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:22.869285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1114/2217] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:22 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:22.874535 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f5fc8-4fd4-4ae6-9ca7-938c6f60f24e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:22.885044 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f5fc8-4fd4-4ae6-9ca7-938c6f60f24e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:22.939746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35e605-96ae-4121-b483-4f30a86c01fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:22.940346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35e605-96ae-4121-b483-4f30a86c01fc admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.940623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35e605-96ae-4121-b483-4f30a86c01fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.940892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35e605-96ae-4121-b483-4f30a86c01fc admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.941526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35e605-96ae-4121-b483-4f30a86c01fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:22.943477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35e605-96ae-4121-b483-4f30a86c01fc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:22.964444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1104/2218] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:22 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:22.968569 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf916b-aad9-4496-98d4-f2d6f080a73a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:22.978627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf916b-aad9-4496-98d4-f2d6f080a73a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:22.983285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e3d35-2cf5-4688-9cf5-8a26b5173024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:22.983867 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e3d35-2cf5-4688-9cf5-8a26b5173024 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:22.984140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e3d35-2cf5-4688-9cf5-8a26b5173024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:22.984433 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e3d35-2cf5-4688-9cf5-8a26b5173024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:22.985064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e3d35-2cf5-4688-9cf5-8a26b5173024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:22.986668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e3d35-2cf5-4688-9cf5-8a26b5173024 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:22.997613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1115/2219] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:23.001612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35e605-96ae-4121-b483-4f30a86c01fc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:23.011550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35e605-96ae-4121-b483-4f30a86c01fc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:23.016907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577ae1b-b496-4aad-adf7-b4fde8fb5615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:23.017558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ae1b-b496-4aad-adf7-b4fde8fb5615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:23.017837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ae1b-b496-4aad-adf7-b4fde8fb5615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:23.018106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ae1b-b496-4aad-adf7-b4fde8fb5615 admin admin] PATH_INFO: `/v3/projects/0bdaee8e1e084e259fa4660fbfc6438b/users/74d6154f47194aa18754fd246df71bc8/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:23.033638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577ae1b-b496-4aad-adf7-b4fde8fb5615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bdaee8e1e084e259fa4660fbfc6438b, user_id=74d6154f47194aa18754fd246df71bc8, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:23.036448 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577ae1b-b496-4aad-adf7-b4fde8fb5615 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:23.050070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1105/2220] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:23 2022] PUT /identity/v3/projects/0bdaee8e1e084e259fa4660fbfc6438b/users/74d6154f47194aa18754fd246df71bc8/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:52:23.053972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e3d35-2cf5-4688-9cf5-8a26b5173024 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:23.064439 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e3d35-2cf5-4688-9cf5-8a26b5173024 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:23.120873 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50fd40-3694-4f23-a112-cacae985eab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:23.121397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50fd40-3694-4f23-a112-cacae985eab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:23.122511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50fd40-3694-4f23-a112-cacae985eab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:23.122836 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50fd40-3694-4f23-a112-cacae985eab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:23.123471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50fd40-3694-4f23-a112-cacae985eab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:23.125103 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50fd40-3694-4f23-a112-cacae985eab6 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:23.134697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1116/2221] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:23.138858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577ae1b-b496-4aad-adf7-b4fde8fb5615 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:23.148865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577ae1b-b496-4aad-adf7-b4fde8fb5615 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:23.153583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103a4df-e790-4d30-8f26-53d07607330a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:23.154253 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103a4df-e790-4d30-8f26-53d07607330a admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:23.154530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103a4df-e790-4d30-8f26-53d07607330a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:23.154801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103a4df-e790-4d30-8f26-53d07607330a admin admin] PATH_INFO: `/v3/projects/0bdaee8e1e084e259fa4660fbfc6438b/users/74d6154f47194aa18754fd246df71bc8/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:23.156955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103a4df-e790-4d30-8f26-53d07607330a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bdaee8e1e084e259fa4660fbfc6438b, user_id=74d6154f47194aa18754fd246df71bc8, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:23.159594 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103a4df-e790-4d30-8f26-53d07607330a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:23.170053 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1106/2222] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:23 2022] PUT /identity/v3/projects/0bdaee8e1e084e259fa4660fbfc6438b/users/74d6154f47194aa18754fd246df71bc8/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:23.181143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f430f7e-df5e-4f63-85f3-98bb6ba094be None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:23.181649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f430f7e-df5e-4f63-85f3-98bb6ba094be None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:23.181766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f430f7e-df5e-4f63-85f3-98bb6ba094be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:23.210156 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0f430f7e-df5e-4f63-85f3-98bb6ba094be None None] MFA Rules not processed for user `74d6154f47194aa18754fd246df71bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:23.211431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0f430f7e-df5e-4f63-85f3-98bb6ba094be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:23.244206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:23.278592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1117/2223] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:23.290226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103a4df-e790-4d30-8f26-53d07607330a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:23.291925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5103a4df-e790-4d30-8f26-53d07607330a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:23.348638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103a4df-e790-4d30-8f26-53d07607330a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:23.352256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fa577-4607-4f0d-8d91-517af0942df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:23.352857 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa577-4607-4f0d-8d91-517af0942df5 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:23.353132 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa577-4607-4f0d-8d91-517af0942df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:23.353401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa577-4607-4f0d-8d91-517af0942df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:23.362954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fa577-4607-4f0d-8d91-517af0942df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:23.364374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fa577-4607-4f0d-8d91-517af0942df5 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:23.374621 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1107/2224] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:23.451256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f430f7e-df5e-4f63-85f3-98bb6ba094be None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:23.463366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f430f7e-df5e-4f63-85f3-98bb6ba094be None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:23.470677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c616-f284-4f06-bc91-395669eb8ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:23.471348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c616-f284-4f06-bc91-395669eb8ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:23.471629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c616-f284-4f06-bc91-395669eb8ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:23.471899 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c616-f284-4f06-bc91-395669eb8ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:23.533398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154c616-f284-4f06-bc91-395669eb8ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:23.534861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154c616-f284-4f06-bc91-395669eb8ab4 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:23.545492 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1118/2225] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:32.982531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fa577-4607-4f0d-8d91-517af0942df5 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:32.985496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-234fa577-4607-4f0d-8d91-517af0942df5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:33.046480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:33.068542 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fa577-4607-4f0d-8d91-517af0942df5 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:33.073951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2100eeb-2b7a-482f-a053-2d0af6aa44db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:33.074595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2100eeb-2b7a-482f-a053-2d0af6aa44db service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:33.074873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2100eeb-2b7a-482f-a053-2d0af6aa44db service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:33.075143 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2100eeb-2b7a-482f-a053-2d0af6aa44db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:33.076970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c2100eeb-2b7a-482f-a053-2d0af6aa44db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:33.088857 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2100eeb-2b7a-482f-a053-2d0af6aa44db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:33.090451 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2100eeb-2b7a-482f-a053-2d0af6aa44db service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:33.100888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1108/2226] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:33.540641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=41 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:34.901511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c616-f284-4f06-bc91-395669eb8ab4 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:34.964201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c616-f284-4f06-bc91-395669eb8ab4 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:34.969972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd43508-b7f8-4304-bafc-cfbeb5ede159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:34.970619 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd43508-b7f8-4304-bafc-cfbeb5ede159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:34.970895 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd43508-b7f8-4304-bafc-cfbeb5ede159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:34.971164 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd43508-b7f8-4304-bafc-cfbeb5ede159 admin admin] PATH_INFO: `/v3/users/65db05a7965945388af6dca10d9aa12f` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:34.972255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd43508-b7f8-4304-bafc-cfbeb5ede159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65db05a7965945388af6dca10d9aa12f)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:34.974281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd43508-b7f8-4304-bafc-cfbeb5ede159 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:35.023596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2bd43508-b7f8-4304-bafc-cfbeb5ede159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65db05a7965945388af6dca10d9aa12f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.024251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2bd43508-b7f8-4304-bafc-cfbeb5ede159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65db05a7965945388af6dca10d9aa12f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.037315 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2bd43508-b7f8-4304-bafc-cfbeb5ede159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65db05a7965945388af6dca10d9aa12f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.037945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2bd43508-b7f8-4304-bafc-cfbeb5ede159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65db05a7965945388af6dca10d9aa12f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.045297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2bd43508-b7f8-4304-bafc-cfbeb5ede159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65db05a7965945388af6dca10d9aa12f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.057836 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1119/2227] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:34 2022] DELETE /identity/v3/users/65db05a7965945388af6dca10d9aa12f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:35.062372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babaed3-77ca-48aa-bd66-7015cce7e8dc None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:35.062872 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babaed3-77ca-48aa-bd66-7015cce7e8dc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:35.062975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babaed3-77ca-48aa-bd66-7015cce7e8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:35.083721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1babaed3-77ca-48aa-bd66-7015cce7e8dc None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:35.085054 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1babaed3-77ca-48aa-bd66-7015cce7e8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:35.112823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd924cc-b220-4b2e-ad51-7ed33465d8b3 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:35.113194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd924cc-b220-4b2e-ad51-7ed33465d8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:35.113499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd924cc-b220-4b2e-ad51-7ed33465d8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:35.132664 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ebd924cc-b220-4b2e-ad51-7ed33465d8b3 None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:35.133860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ebd924cc-b220-4b2e-ad51-7ed33465d8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:35.145388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1109/2228] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:35.156343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babaed3-77ca-48aa-bd66-7015cce7e8dc None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:35.157802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1babaed3-77ca-48aa-bd66-7015cce7e8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:35.191117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1120/2229] 10.222.0.20 () {58 vars in 967 bytes} [Mon Oct 24 13:52:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:52:35.205663 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd924cc-b220-4b2e-ad51-7ed33465d8b3 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:35.207075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ebd924cc-b220-4b2e-ad51-7ed33465d8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:35.220995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babaed3-77ca-48aa-bd66-7015cce7e8dc None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:35.224966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534a8b5-cb33-461c-9407-c11600c8514e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:35.225832 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534a8b5-cb33-461c-9407-c11600c8514e service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:35.226074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534a8b5-cb33-461c-9407-c11600c8514e service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:35.226432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534a8b5-cb33-461c-9407-c11600c8514e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:35.236060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534a8b5-cb33-461c-9407-c11600c8514e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:35.237560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534a8b5-cb33-461c-9407-c11600c8514e service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:35.240641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:35.248824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1110/2230] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:52:35.288653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd924cc-b220-4b2e-ad51-7ed33465d8b3 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:35.298807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cf494-8b27-426c-a451-544faef1b12c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:35.299585 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cf494-8b27-426c-a451-544faef1b12c service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:35.299965 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cf494-8b27-426c-a451-544faef1b12c service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:35.300449 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cf494-8b27-426c-a451-544faef1b12c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:35.309403 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cf494-8b27-426c-a451-544faef1b12c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:35.311334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cf494-8b27-426c-a451-544faef1b12c service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:35.321309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1121/2231] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:52:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:52:35.635574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534a8b5-cb33-461c-9407-c11600c8514e service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:35.636985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2534a8b5-cb33-461c-9407-c11600c8514e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:35.658588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534a8b5-cb33-461c-9407-c11600c8514e service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:35.664110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:35.664978 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:35.665428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:35.665918 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] PATH_INFO: `/v3/projects/66f8abe48c7444689eacae538b5468d2` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:35.667546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f8abe48c7444689eacae538b5468d2)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:35.670219 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:35.719314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f8abe48c7444689eacae538b5468d2', 'request_id': 'req-799fbba1-e3b4-4717-8426-35a41897c429'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.727016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f8abe48c7444689eacae538b5468d2', 'request_id': 'req-799fbba1-e3b4-4717-8426-35a41897c429'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.733607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f8abe48c7444689eacae538b5468d2', 'request_id': 'req-799fbba1-e3b4-4717-8426-35a41897c429'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.744047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] The token cache is being invalidate because project 66f8abe48c7444689eacae538b5468d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:35.744333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.744991 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.746510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1111/2232] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:35 2022] DELETE /identity/v3/projects/66f8abe48c7444689eacae538b5468d2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:35.751637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cf494-8b27-426c-a451-544faef1b12c service swift] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:35.763642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cf494-8b27-426c-a451-544faef1b12c service swift] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:35.819238 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134755ad-3d9e-4a56-89bd-7a64293ba3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:35.820533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134755ad-3d9e-4a56-89bd-7a64293ba3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:35.820812 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134755ad-3d9e-4a56-89bd-7a64293ba3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:35.821082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134755ad-3d9e-4a56-89bd-7a64293ba3bf admin admin] PATH_INFO: `/v3/users/74d6154f47194aa18754fd246df71bc8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:35.822407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134755ad-3d9e-4a56-89bd-7a64293ba3bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d6154f47194aa18754fd246df71bc8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:35.824318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134755ad-3d9e-4a56-89bd-7a64293ba3bf admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:35.872979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-134755ad-3d9e-4a56-89bd-7a64293ba3bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d6154f47194aa18754fd246df71bc8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.873645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-134755ad-3d9e-4a56-89bd-7a64293ba3bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d6154f47194aa18754fd246df71bc8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.886952 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-134755ad-3d9e-4a56-89bd-7a64293ba3bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d6154f47194aa18754fd246df71bc8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.887616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-134755ad-3d9e-4a56-89bd-7a64293ba3bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d6154f47194aa18754fd246df71bc8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.896319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-134755ad-3d9e-4a56-89bd-7a64293ba3bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d6154f47194aa18754fd246df71bc8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:35.909294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1122/2233] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:35 2022] DELETE /identity/v3/users/74d6154f47194aa18754fd246df71bc8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:36.182413 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:36.183865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:36.197629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fbba1-e3b4-4717-8426-35a41897c429 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:36.203980 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:36.260643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:36.261297 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:36.261574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:36.261843 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] PATH_INFO: `/v3/projects/0bdaee8e1e084e259fa4660fbfc6438b` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:36.263091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bdaee8e1e084e259fa4660fbfc6438b)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:36.264980 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:36.310888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bdaee8e1e084e259fa4660fbfc6438b', 'request_id': 'req-9eded027-a5f9-46de-b45f-08e54272272e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:36.317934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bdaee8e1e084e259fa4660fbfc6438b', 'request_id': 'req-9eded027-a5f9-46de-b45f-08e54272272e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:36.324275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bdaee8e1e084e259fa4660fbfc6438b', 'request_id': 'req-9eded027-a5f9-46de-b45f-08e54272272e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:36.334458 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] The token cache is being invalidate because project 0bdaee8e1e084e259fa4660fbfc6438b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:36.334739 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:36.335356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-9eded027-a5f9-46de-b45f-08e54272272e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:36.336806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1112/2234] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:36 2022] DELETE /identity/v3/projects/0bdaee8e1e084e259fa4660fbfc6438b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:36.342280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f5de7-69ee-4e06-a8ff-3ac46987c625 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:36.342557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f5de7-69ee-4e06-a8ff-3ac46987c625 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:36.342819 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f5de7-69ee-4e06-a8ff-3ac46987c625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:36.362927 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-459f5de7-69ee-4e06-a8ff-3ac46987c625 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:36.364115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-459f5de7-69ee-4e06-a8ff-3ac46987c625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:36.394368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:36.427390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1123/2235] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:36.435417 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb0616-06cb-4541-aa6f-e2cc482b965d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:36.435699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb0616-06cb-4541-aa6f-e2cc482b965d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:36.435969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb0616-06cb-4541-aa6f-e2cc482b965d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:36.456318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-2dcb0616-06cb-4541-aa6f-e2cc482b965d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:36.457505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2dcb0616-06cb-4541-aa6f-e2cc482b965d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:36.465354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1113/2236] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:36.470186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f5de7-69ee-4e06-a8ff-3ac46987c625 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:36.480834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f5de7-69ee-4e06-a8ff-3ac46987c625 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:36.486410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11481d55-37a9-4d57-8b06-319a5e5094b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:36.488443 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11481d55-37a9-4d57-8b06-319a5e5094b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:36.488726 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11481d55-37a9-4d57-8b06-319a5e5094b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:36.489001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11481d55-37a9-4d57-8b06-319a5e5094b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:36.489657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11481d55-37a9-4d57-8b06-319a5e5094b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:52:36.489958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11481d55-37a9-4d57-8b06-319a5e5094b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:36.491574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11481d55-37a9-4d57-8b06-319a5e5094b1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:36.503740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1124/2237] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:52:36.508064 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb0616-06cb-4541-aa6f-e2cc482b965d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:36.517487 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb0616-06cb-4541-aa6f-e2cc482b965d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:36.522078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cb9aa-17cf-496a-aad7-38c98487d91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:36.522741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cb9aa-17cf-496a-aad7-38c98487d91e admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:36.523020 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cb9aa-17cf-496a-aad7-38c98487d91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:36.523292 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cb9aa-17cf-496a-aad7-38c98487d91e admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:36.523952 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15cb9aa-17cf-496a-aad7-38c98487d91e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:36.525834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15cb9aa-17cf-496a-aad7-38c98487d91e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:36.555140 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1114/2238] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:36 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:52:36.559302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11481d55-37a9-4d57-8b06-319a5e5094b1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:36.569678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11481d55-37a9-4d57-8b06-319a5e5094b1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:36.627210 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918318f-c127-45d2-aa93-639362808086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:36.627812 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918318f-c127-45d2-aa93-639362808086 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:36.628088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918318f-c127-45d2-aa93-639362808086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:36.628355 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918318f-c127-45d2-aa93-639362808086 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:36.628972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918318f-c127-45d2-aa93-639362808086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:36.630890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918318f-c127-45d2-aa93-639362808086 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:36.654369 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1125/2239] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:36 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:52:36.658671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cb9aa-17cf-496a-aad7-38c98487d91e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:36.668701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cb9aa-17cf-496a-aad7-38c98487d91e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:36.673327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec3095-cb13-4ab4-9bdc-a287dd4235fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:36.673945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec3095-cb13-4ab4-9bdc-a287dd4235fb admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:36.674258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec3095-cb13-4ab4-9bdc-a287dd4235fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:36.674530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec3095-cb13-4ab4-9bdc-a287dd4235fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:36.675164 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec3095-cb13-4ab4-9bdc-a287dd4235fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:36.676727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec3095-cb13-4ab4-9bdc-a287dd4235fb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:36.687004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1115/2240] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:36.690858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918318f-c127-45d2-aa93-639362808086 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:36.701263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918318f-c127-45d2-aa93-639362808086 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:36.705858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32e107-cc6b-4c37-869c-4055a7e87633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:36.706489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32e107-cc6b-4c37-869c-4055a7e87633 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:36.706766 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32e107-cc6b-4c37-869c-4055a7e87633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:36.707038 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32e107-cc6b-4c37-869c-4055a7e87633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:36.707676 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32e107-cc6b-4c37-869c-4055a7e87633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:36.709275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32e107-cc6b-4c37-869c-4055a7e87633 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:36.720334 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1126/2241] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:36.724276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec3095-cb13-4ab4-9bdc-a287dd4235fb admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:36.734112 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec3095-cb13-4ab4-9bdc-a287dd4235fb admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:36.738693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c3ef9-fa54-46ed-86c1-14d174fa491b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:36.739324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c3ef9-fa54-46ed-86c1-14d174fa491b admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:36.739603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c3ef9-fa54-46ed-86c1-14d174fa491b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:36.739870 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c3ef9-fa54-46ed-86c1-14d174fa491b admin admin] PATH_INFO: `/v3/projects/44f6aa04a3b04d9e9180e2c9654b6d09/users/bc81433cd5024ea1bd73361af2cef5fc/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:36.753948 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c3ef9-fa54-46ed-86c1-14d174fa491b admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f6aa04a3b04d9e9180e2c9654b6d09, user_id=bc81433cd5024ea1bd73361af2cef5fc, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:36.756635 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c3ef9-fa54-46ed-86c1-14d174fa491b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:36.770420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1116/2242] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:36 2022] PUT /identity/v3/projects/44f6aa04a3b04d9e9180e2c9654b6d09/users/bc81433cd5024ea1bd73361af2cef5fc/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:52:36.775435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66116042-3882-4edb-9422-0c0af9833e22 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:36.775718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66116042-3882-4edb-9422-0c0af9833e22 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:36.775984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66116042-3882-4edb-9422-0c0af9833e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:36.795280 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-66116042-3882-4edb-9422-0c0af9833e22 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:36.796449 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-66116042-3882-4edb-9422-0c0af9833e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:36.854329 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1127/2243] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:36.863911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c3ef9-fa54-46ed-86c1-14d174fa491b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:36.865658 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2e1c3ef9-fa54-46ed-86c1-14d174fa491b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:36.900471 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:36.924716 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c3ef9-fa54-46ed-86c1-14d174fa491b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:36.928400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b7344-7c34-4068-b8b7-08bb19fd2a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:36.928998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b7344-7c34-4068-b8b7-08bb19fd2a15 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:36.929276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b7344-7c34-4068-b8b7-08bb19fd2a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:36.929546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b7344-7c34-4068-b8b7-08bb19fd2a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:36.938818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b7344-7c34-4068-b8b7-08bb19fd2a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:36.940236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b7344-7c34-4068-b8b7-08bb19fd2a15 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:36.949552 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1117/2244] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:37.579361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabd86e-d889-4529-8707-47b6b6177c3b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:37.579875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabd86e-d889-4529-8707-47b6b6177c3b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:37.579996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabd86e-d889-4529-8707-47b6b6177c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:37.608558 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-ffabd86e-d889-4529-8707-47b6b6177c3b None None] MFA Rules not processed for user `bc81433cd5024ea1bd73361af2cef5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:37.609763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ffabd86e-d889-4529-8707-47b6b6177c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:37.639503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:37.679662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1128/2245] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:37.684021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b7344-7c34-4068-b8b7-08bb19fd2a15 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:37.694576 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b7344-7c34-4068-b8b7-08bb19fd2a15 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:37.699092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ec88a-f9a6-4c57-a8eb-0a45ab902670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:37.699683 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ec88a-f9a6-4c57-a8eb-0a45ab902670 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:37.699964 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ec88a-f9a6-4c57-a8eb-0a45ab902670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:37.700390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ec88a-f9a6-4c57-a8eb-0a45ab902670 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:37.701341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ec88a-f9a6-4c57-a8eb-0a45ab902670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:37.703285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ec88a-f9a6-4c57-a8eb-0a45ab902670 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:37.734156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1118/2246] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:37 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:52:37.735431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabd86e-d889-4529-8707-47b6b6177c3b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:37.746198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabd86e-d889-4529-8707-47b6b6177c3b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:37.803878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5f7de-36af-4d4f-90c3-e4e964719903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:37.804518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f7de-36af-4d4f-90c3-e4e964719903 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:37.804803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f7de-36af-4d4f-90c3-e4e964719903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:37.805075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f7de-36af-4d4f-90c3-e4e964719903 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:37.805704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5f7de-36af-4d4f-90c3-e4e964719903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:37.807668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5f7de-36af-4d4f-90c3-e4e964719903 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:37.828927 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1129/2247] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:37 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:37.832941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ec88a-f9a6-4c57-a8eb-0a45ab902670 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:37.843694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ec88a-f9a6-4c57-a8eb-0a45ab902670 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:37.848273 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215da836-4154-4802-afb2-0fd6bba02603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:37.848860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215da836-4154-4802-afb2-0fd6bba02603 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:37.849137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215da836-4154-4802-afb2-0fd6bba02603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:37.849405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215da836-4154-4802-afb2-0fd6bba02603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:37.850037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215da836-4154-4802-afb2-0fd6bba02603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:37.851669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215da836-4154-4802-afb2-0fd6bba02603 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:37.862972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1119/2248] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:37.867344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5f7de-36af-4d4f-90c3-e4e964719903 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:37.877255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5f7de-36af-4d4f-90c3-e4e964719903 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:37.881790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09df53-a893-402c-ac79-d9de48258d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:37.882469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09df53-a893-402c-ac79-d9de48258d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:37.882745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09df53-a893-402c-ac79-d9de48258d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:37.883015 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09df53-a893-402c-ac79-d9de48258d5d admin admin] PATH_INFO: `/v3/projects/3349f60b943742cbb09403eea109fd07/users/bafd51603b244943963777bc16090840/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:37.898727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09df53-a893-402c-ac79-d9de48258d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3349f60b943742cbb09403eea109fd07, user_id=bafd51603b244943963777bc16090840, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:37.901374 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09df53-a893-402c-ac79-d9de48258d5d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:37.915101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1130/2249] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:37 2022] PUT /identity/v3/projects/3349f60b943742cbb09403eea109fd07/users/bafd51603b244943963777bc16090840/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:52:37.919675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215da836-4154-4802-afb2-0fd6bba02603 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:37.929509 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215da836-4154-4802-afb2-0fd6bba02603 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:37.985341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52351216-1578-4857-a495-2828649dac0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:37.985895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52351216-1578-4857-a495-2828649dac0a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:37.986222 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52351216-1578-4857-a495-2828649dac0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:37.986576 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52351216-1578-4857-a495-2828649dac0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:37.987320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52351216-1578-4857-a495-2828649dac0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:37.988900 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52351216-1578-4857-a495-2828649dac0a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:37.998950 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1120/2250] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:38.003268 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09df53-a893-402c-ac79-d9de48258d5d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:38.013597 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09df53-a893-402c-ac79-d9de48258d5d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:38.018261 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddd304-c6ed-4a90-8695-48846a8c181d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:38.018929 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddd304-c6ed-4a90-8695-48846a8c181d admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:38.019205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddd304-c6ed-4a90-8695-48846a8c181d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:38.019467 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddd304-c6ed-4a90-8695-48846a8c181d admin admin] PATH_INFO: `/v3/projects/3349f60b943742cbb09403eea109fd07/users/bafd51603b244943963777bc16090840/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:38.021370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ddd304-c6ed-4a90-8695-48846a8c181d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3349f60b943742cbb09403eea109fd07, user_id=bafd51603b244943963777bc16090840, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:38.024017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ddd304-c6ed-4a90-8695-48846a8c181d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:38.035170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1131/2251] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:38 2022] PUT /identity/v3/projects/3349f60b943742cbb09403eea109fd07/users/bafd51603b244943963777bc16090840/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:38.670647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e13fb-96cf-4271-815d-42943b6b186b None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:38.670928 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e13fb-96cf-4271-815d-42943b6b186b None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:38.671226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e13fb-96cf-4271-815d-42943b6b186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:38.708121 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1d1e13fb-96cf-4271-815d-42943b6b186b None None] MFA Rules not processed for user `bafd51603b244943963777bc16090840`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:38.709330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1d1e13fb-96cf-4271-815d-42943b6b186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:38.792799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1121/2252] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:52:38.799975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddd304-c6ed-4a90-8695-48846a8c181d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:38.803133 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b4ddd304-c6ed-4a90-8695-48846a8c181d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:38.869178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddd304-c6ed-4a90-8695-48846a8c181d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:38.873320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb90b7-1528-4665-ba25-eecf3f34ba06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:38.873923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb90b7-1528-4665-ba25-eecf3f34ba06 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:38.874290 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb90b7-1528-4665-ba25-eecf3f34ba06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:38.874562 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb90b7-1528-4665-ba25-eecf3f34ba06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:38.913770 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:38.942717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb90b7-1528-4665-ba25-eecf3f34ba06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:38.944147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb90b7-1528-4665-ba25-eecf3f34ba06 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:38.958922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1132/2253] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:40.325080 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e13fb-96cf-4271-815d-42943b6b186b None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:40.327228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1d1e13fb-96cf-4271-815d-42943b6b186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:40.339247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e13fb-96cf-4271-815d-42943b6b186b None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:40.343331 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e76b8b-4e18-4c8c-a2a4-d7adb7273a53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:40.343939 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e76b8b-4e18-4c8c-a2a4-d7adb7273a53 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:40.344193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e76b8b-4e18-4c8c-a2a4-d7adb7273a53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:40.344461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e76b8b-4e18-4c8c-a2a4-d7adb7273a53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:40.354126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e76b8b-4e18-4c8c-a2a4-d7adb7273a53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:40.355560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e76b8b-4e18-4c8c-a2a4-d7adb7273a53 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:40.366028 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1122/2254] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:48.056521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb90b7-1528-4665-ba25-eecf3f34ba06 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:48.109878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb90b7-1528-4665-ba25-eecf3f34ba06 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:48.113492 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df922f-32a6-472e-9627-9a3e3171a7a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:48.114086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df922f-32a6-472e-9627-9a3e3171a7a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:48.114389 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df922f-32a6-472e-9627-9a3e3171a7a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:48.114656 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df922f-32a6-472e-9627-9a3e3171a7a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:48.116349 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-96df922f-32a6-472e-9627-9a3e3171a7a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:48.181281 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df922f-32a6-472e-9627-9a3e3171a7a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:48.182825 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df922f-32a6-472e-9627-9a3e3171a7a8 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:48.193479 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1133/2255] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:50.362225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:51.761094 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e76b8b-4e18-4c8c-a2a4-d7adb7273a53 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:51.762577 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b4e76b8b-4e18-4c8c-a2a4-d7adb7273a53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:51.826885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df922f-32a6-472e-9627-9a3e3171a7a8 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:51.829183 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-96df922f-32a6-472e-9627-9a3e3171a7a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:51.844549 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df922f-32a6-472e-9627-9a3e3171a7a8 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:51.849358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fe0e8-e0cf-4483-9a29-62fa33ba912f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:51.850011 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fe0e8-e0cf-4483-9a29-62fa33ba912f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:51.850331 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fe0e8-e0cf-4483-9a29-62fa33ba912f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:51.850636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fe0e8-e0cf-4483-9a29-62fa33ba912f admin admin] PATH_INFO: `/v3/users/1461a94dd8ef48f79a0d8642a1bc1e58` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:51.852905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e76b8b-4e18-4c8c-a2a4-d7adb7273a53 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:51.858798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28d4eb-34eb-426d-92e2-678ad29e4461 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:51.859503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28d4eb-34eb-426d-92e2-678ad29e4461 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:51.859853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28d4eb-34eb-426d-92e2-678ad29e4461 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:51.860186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28d4eb-34eb-426d-92e2-678ad29e4461 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:51.862010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1a28d4eb-34eb-426d-92e2-678ad29e4461 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:51.865776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fe0e8-e0cf-4483-9a29-62fa33ba912f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1461a94dd8ef48f79a0d8642a1bc1e58)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:51.867749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fe0e8-e0cf-4483-9a29-62fa33ba912f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:51.874957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28d4eb-34eb-426d-92e2-678ad29e4461 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:51.876534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28d4eb-34eb-426d-92e2-678ad29e4461 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:51.887079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1123/2256] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:51.919577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3e2fe0e8-e0cf-4483-9a29-62fa33ba912f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1461a94dd8ef48f79a0d8642a1bc1e58'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:51.920231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3e2fe0e8-e0cf-4483-9a29-62fa33ba912f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1461a94dd8ef48f79a0d8642a1bc1e58'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:51.932710 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3e2fe0e8-e0cf-4483-9a29-62fa33ba912f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1461a94dd8ef48f79a0d8642a1bc1e58'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:51.933357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3e2fe0e8-e0cf-4483-9a29-62fa33ba912f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1461a94dd8ef48f79a0d8642a1bc1e58'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:51.937941 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:51.941151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-3e2fe0e8-e0cf-4483-9a29-62fa33ba912f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1461a94dd8ef48f79a0d8642a1bc1e58'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:51.954159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1134/2257] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:51 2022] DELETE /identity/v3/users/1461a94dd8ef48f79a0d8642a1bc1e58 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:52.129475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28d4eb-34eb-426d-92e2-678ad29e4461 service placement] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:52.140728 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28d4eb-34eb-426d-92e2-678ad29e4461 service placement] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:52.200163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:52.200799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:52.201235 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:52.201487 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] PATH_INFO: `/v3/projects/ae58cf3c8178463296c8b633550a130e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:52.213066 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae58cf3c8178463296c8b633550a130e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:52.215044 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:52.261292 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae58cf3c8178463296c8b633550a130e', 'request_id': 'req-d3803221-f5fd-43a6-a8c4-e9df19166483'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:52.268565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae58cf3c8178463296c8b633550a130e', 'request_id': 'req-d3803221-f5fd-43a6-a8c4-e9df19166483'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:52.275199 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae58cf3c8178463296c8b633550a130e', 'request_id': 'req-d3803221-f5fd-43a6-a8c4-e9df19166483'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:52.286482 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] The token cache is being invalidate because project ae58cf3c8178463296c8b633550a130e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:52.286798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:52.287429 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d3803221-f5fd-43a6-a8c4-e9df19166483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:52.289078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1124/2258] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:52 2022] DELETE /identity/v3/projects/ae58cf3c8178463296c8b633550a130e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:52.294013 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef1c87-49d5-4e34-8083-47cf7273ad92 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:52.294368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef1c87-49d5-4e34-8083-47cf7273ad92 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:52.294641 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef1c87-49d5-4e34-8083-47cf7273ad92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:52.316362 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-82ef1c87-49d5-4e34-8083-47cf7273ad92 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:52.317745 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-82ef1c87-49d5-4e34-8083-47cf7273ad92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:52.376668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1135/2259] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:52.384520 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393701e9-2233-4bb6-b8c2-656878a23310 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:52.384809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393701e9-2233-4bb6-b8c2-656878a23310 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:52.385083 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393701e9-2233-4bb6-b8c2-656878a23310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:52.406248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-393701e9-2233-4bb6-b8c2-656878a23310 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:52.407528 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-393701e9-2233-4bb6-b8c2-656878a23310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:52.415380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1125/2260] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:52.419549 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef1c87-49d5-4e34-8083-47cf7273ad92 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:52.429979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef1c87-49d5-4e34-8083-47cf7273ad92 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:52.434465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c91ff1-3143-419b-a557-a68071656a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:52.435055 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c91ff1-3143-419b-a557-a68071656a32 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:52.435355 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c91ff1-3143-419b-a557-a68071656a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:52.435621 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c91ff1-3143-419b-a557-a68071656a32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:52.436875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c91ff1-3143-419b-a557-a68071656a32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:52:52.437419 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c91ff1-3143-419b-a557-a68071656a32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:52.439061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c91ff1-3143-419b-a557-a68071656a32 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:52.450343 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1136/2261] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:52:52.454727 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393701e9-2233-4bb6-b8c2-656878a23310 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:52.460281 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:52.464155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393701e9-2233-4bb6-b8c2-656878a23310 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:52.470575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf1851-a2bd-4b1a-bf4a-d018572114b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:52.471334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf1851-a2bd-4b1a-bf4a-d018572114b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:52.471785 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf1851-a2bd-4b1a-bf4a-d018572114b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:52.472116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf1851-a2bd-4b1a-bf4a-d018572114b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:52.472882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf1851-a2bd-4b1a-bf4a-d018572114b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:52.474920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf1851-a2bd-4b1a-bf4a-d018572114b1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:52.502911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1126/2262] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:52 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:52.507069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c91ff1-3143-419b-a557-a68071656a32 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:52.517395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c91ff1-3143-419b-a557-a68071656a32 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:52.572262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2460c-c6a6-4ae1-8c79-1da6f4d92741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:52.572862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2460c-c6a6-4ae1-8c79-1da6f4d92741 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:52.573141 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2460c-c6a6-4ae1-8c79-1da6f4d92741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:52.573412 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2460c-c6a6-4ae1-8c79-1da6f4d92741 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:52.574030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2460c-c6a6-4ae1-8c79-1da6f4d92741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:52.575959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2460c-c6a6-4ae1-8c79-1da6f4d92741 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:52.597380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1137/2263] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:52 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:52.601498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf1851-a2bd-4b1a-bf4a-d018572114b1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:52.612686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf1851-a2bd-4b1a-bf4a-d018572114b1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:52.617348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bcc33-5de5-4e28-b4f1-94e0422b00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:52.618019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bcc33-5de5-4e28-b4f1-94e0422b00bd admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:52.618325 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bcc33-5de5-4e28-b4f1-94e0422b00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:52.618593 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bcc33-5de5-4e28-b4f1-94e0422b00bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:52.619229 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bcc33-5de5-4e28-b4f1-94e0422b00bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:52.620799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bcc33-5de5-4e28-b4f1-94e0422b00bd admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:52.632494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1127/2264] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:52.636045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2460c-c6a6-4ae1-8c79-1da6f4d92741 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:52.646834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2460c-c6a6-4ae1-8c79-1da6f4d92741 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:52.651613 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea999d-7a7c-45c5-a4df-6c3ff9eb6c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:52.652250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea999d-7a7c-45c5-a4df-6c3ff9eb6c0f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:52.652481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea999d-7a7c-45c5-a4df-6c3ff9eb6c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:52.652753 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea999d-7a7c-45c5-a4df-6c3ff9eb6c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:52.653427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea999d-7a7c-45c5-a4df-6c3ff9eb6c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:52.655215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea999d-7a7c-45c5-a4df-6c3ff9eb6c0f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:52.669632 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1138/2265] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:52.670888 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bcc33-5de5-4e28-b4f1-94e0422b00bd admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:52.680910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bcc33-5de5-4e28-b4f1-94e0422b00bd admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:52.685736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692c840-2daa-40a8-a8d0-59da79f54143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:52.686951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692c840-2daa-40a8-a8d0-59da79f54143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:52.687350 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692c840-2daa-40a8-a8d0-59da79f54143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:52.687704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692c840-2daa-40a8-a8d0-59da79f54143 admin admin] PATH_INFO: `/v3/projects/b38e8307e3d94032845d8098b624bc80/users/8a81f70baa8b43f9b310cb5d661a8aad/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:52.703046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692c840-2daa-40a8-a8d0-59da79f54143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38e8307e3d94032845d8098b624bc80, user_id=8a81f70baa8b43f9b310cb5d661a8aad, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:52.705689 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692c840-2daa-40a8-a8d0-59da79f54143 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:52.720445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1128/2266] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:52 2022] PUT /identity/v3/projects/b38e8307e3d94032845d8098b624bc80/users/8a81f70baa8b43f9b310cb5d661a8aad/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:52.732034 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a708682-e051-4c69-8ce9-e728204aea58 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:52.732325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a708682-e051-4c69-8ce9-e728204aea58 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:52.732601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a708682-e051-4c69-8ce9-e728204aea58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:52.760643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-2a708682-e051-4c69-8ce9-e728204aea58 None None] MFA Rules not processed for user `8a81f70baa8b43f9b310cb5d661a8aad`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:52.761853 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2a708682-e051-4c69-8ce9-e728204aea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:52.807821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:52.835974 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1139/2267] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:52.837404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692c840-2daa-40a8-a8d0-59da79f54143 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:52.901568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692c840-2daa-40a8-a8d0-59da79f54143 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:52.907484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5396c04-bccc-42b8-87f7-3ee51b3259ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:52.908181 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5396c04-bccc-42b8-87f7-3ee51b3259ba admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:52.908519 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5396c04-bccc-42b8-87f7-3ee51b3259ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:52.908736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5396c04-bccc-42b8-87f7-3ee51b3259ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:52.909373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5396c04-bccc-42b8-87f7-3ee51b3259ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:52.911314 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5396c04-bccc-42b8-87f7-3ee51b3259ba admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:52.943756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1129/2268] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:52 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:52.946562 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a708682-e051-4c69-8ce9-e728204aea58 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:52.961523 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a708682-e051-4c69-8ce9-e728204aea58 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:53.030845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a8a76-b83a-4ff3-95b4-69b87eb84f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:53.031697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a8a76-b83a-4ff3-95b4-69b87eb84f73 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:53.031978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a8a76-b83a-4ff3-95b4-69b87eb84f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:53.032246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a8a76-b83a-4ff3-95b4-69b87eb84f73 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:53.033028 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a8a76-b83a-4ff3-95b4-69b87eb84f73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:53.035603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a8a76-b83a-4ff3-95b4-69b87eb84f73 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:53.060158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1140/2269] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:52 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:52:53.065016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5396c04-bccc-42b8-87f7-3ee51b3259ba admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:53.076004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5396c04-bccc-42b8-87f7-3ee51b3259ba admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:53.082187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b245ad-8c2a-4a92-98df-50b24f00b468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:53.083007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b245ad-8c2a-4a92-98df-50b24f00b468 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:53.083290 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b245ad-8c2a-4a92-98df-50b24f00b468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:53.083561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b245ad-8c2a-4a92-98df-50b24f00b468 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:53.084210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b245ad-8c2a-4a92-98df-50b24f00b468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:53.085805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b245ad-8c2a-4a92-98df-50b24f00b468 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:53.098514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1130/2270] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:52:53.102588 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a8a76-b83a-4ff3-95b4-69b87eb84f73 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:53.113503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a8a76-b83a-4ff3-95b4-69b87eb84f73 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:53.118688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0488d6-0c67-492d-af2e-84dca2c6a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:53.119360 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0488d6-0c67-492d-af2e-84dca2c6a50f admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:53.119795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0488d6-0c67-492d-af2e-84dca2c6a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:53.120075 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0488d6-0c67-492d-af2e-84dca2c6a50f admin admin] PATH_INFO: `/v3/projects/b5ef571941ab4d5ebd00c4b47bf86f30/users/dbb23e955c1f46db867121dc4861abda/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:53.137859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0488d6-0c67-492d-af2e-84dca2c6a50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ef571941ab4d5ebd00c4b47bf86f30, user_id=dbb23e955c1f46db867121dc4861abda, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:53.141222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0488d6-0c67-492d-af2e-84dca2c6a50f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:53.156941 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1141/2271] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:53 2022] PUT /identity/v3/projects/b5ef571941ab4d5ebd00c4b47bf86f30/users/dbb23e955c1f46db867121dc4861abda/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:53.160654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b245ad-8c2a-4a92-98df-50b24f00b468 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:53.171212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b245ad-8c2a-4a92-98df-50b24f00b468 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:53.234087 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3a5f5-dd17-4d2e-9bc7-3b78afef4f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:53.234841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3a5f5-dd17-4d2e-9bc7-3b78afef4f50 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:53.235103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3a5f5-dd17-4d2e-9bc7-3b78afef4f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:53.235374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3a5f5-dd17-4d2e-9bc7-3b78afef4f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:53.236017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3a5f5-dd17-4d2e-9bc7-3b78afef4f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:53.237607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3a5f5-dd17-4d2e-9bc7-3b78afef4f50 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:53.250015 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1131/2272] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:53.254437 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0488d6-0c67-492d-af2e-84dca2c6a50f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:53.265963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0488d6-0c67-492d-af2e-84dca2c6a50f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:53.270793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b71dd-419d-49e4-8a77-1a929277fcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:53.271450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b71dd-419d-49e4-8a77-1a929277fcb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:53.271744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b71dd-419d-49e4-8a77-1a929277fcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:53.272567 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b71dd-419d-49e4-8a77-1a929277fcb1 admin admin] PATH_INFO: `/v3/projects/b5ef571941ab4d5ebd00c4b47bf86f30/users/dbb23e955c1f46db867121dc4861abda/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:53.274027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b71dd-419d-49e4-8a77-1a929277fcb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ef571941ab4d5ebd00c4b47bf86f30, user_id=dbb23e955c1f46db867121dc4861abda, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:53.276698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b71dd-419d-49e4-8a77-1a929277fcb1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:53.288645 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1142/2273] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:53 2022] PUT /identity/v3/projects/b5ef571941ab4d5ebd00c4b47bf86f30/users/dbb23e955c1f46db867121dc4861abda/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:53.299527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f562-685d-441e-b5af-c63da9749847 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:53.299826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f562-685d-441e-b5af-c63da9749847 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:53.300096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5f562-685d-441e-b5af-c63da9749847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:53.328959 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-58d5f562-685d-441e-b5af-c63da9749847 None None] MFA Rules not processed for user `dbb23e955c1f46db867121dc4861abda`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:53.330234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-58d5f562-685d-441e-b5af-c63da9749847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:53.335896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:53.404259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1132/2274] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:53.416831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b71dd-419d-49e4-8a77-1a929277fcb1 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:53.418608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-804b71dd-419d-49e4-8a77-1a929277fcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:53.480966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b71dd-419d-49e4-8a77-1a929277fcb1 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:53.485839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05dc3e-dadd-4f67-b8b9-156ec9fda5aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:53.486602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05dc3e-dadd-4f67-b8b9-156ec9fda5aa service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:53.487025 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05dc3e-dadd-4f67-b8b9-156ec9fda5aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:53.487375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05dc3e-dadd-4f67-b8b9-156ec9fda5aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:53.507736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05dc3e-dadd-4f67-b8b9-156ec9fda5aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:53.509345 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05dc3e-dadd-4f67-b8b9-156ec9fda5aa service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:53.519586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1143/2275] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:53.816295 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5f562-685d-441e-b5af-c63da9749847 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:53.817801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-58d5f562-685d-441e-b5af-c63da9749847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:53.890014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5f562-685d-441e-b5af-c63da9749847 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:53.896667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a4068-1c75-40cd-be34-5f4ed279ea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:53.897307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a4068-1c75-40cd-be34-5f4ed279ea82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:53.897583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a4068-1c75-40cd-be34-5f4ed279ea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:53.897856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a4068-1c75-40cd-be34-5f4ed279ea82 admin admin] PATH_INFO: `/v3/users/bc81433cd5024ea1bd73361af2cef5fc` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:53.898996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a4068-1c75-40cd-be34-5f4ed279ea82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc81433cd5024ea1bd73361af2cef5fc)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:53.900883 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a4068-1c75-40cd-be34-5f4ed279ea82 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:53.952926 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8f2a4068-1c75-40cd-be34-5f4ed279ea82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc81433cd5024ea1bd73361af2cef5fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:53.953579 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8f2a4068-1c75-40cd-be34-5f4ed279ea82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc81433cd5024ea1bd73361af2cef5fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:53.954200 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8f2a4068-1c75-40cd-be34-5f4ed279ea82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc81433cd5024ea1bd73361af2cef5fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:53.962791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8f2a4068-1c75-40cd-be34-5f4ed279ea82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc81433cd5024ea1bd73361af2cef5fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:53.969327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05dc3e-dadd-4f67-b8b9-156ec9fda5aa service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:53.978234 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-8f2a4068-1c75-40cd-be34-5f4ed279ea82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc81433cd5024ea1bd73361af2cef5fc'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:53.981837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05dc3e-dadd-4f67-b8b9-156ec9fda5aa service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:53.987870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:53.989509 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:53.989988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:53.990728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] PATH_INFO: `/v3/users/8a81f70baa8b43f9b310cb5d661a8aad` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:53.992592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a81f70baa8b43f9b310cb5d661a8aad)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:53.994596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:53.997347 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1133/2276] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:53 2022] DELETE /identity/v3/users/bc81433cd5024ea1bd73361af2cef5fc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:54.033550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:54.052367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a81f70baa8b43f9b310cb5d661a8aad'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.053627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a81f70baa8b43f9b310cb5d661a8aad'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.071659 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a81f70baa8b43f9b310cb5d661a8aad'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.072434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a81f70baa8b43f9b310cb5d661a8aad'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.079864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a81f70baa8b43f9b310cb5d661a8aad'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.093738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1144/2277] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:53 2022] DELETE /identity/v3/users/8a81f70baa8b43f9b310cb5d661a8aad => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:54.098357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2152ad-9cbd-4fa7-9d83-5c8a3774f3c0 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:54.098646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2152ad-9cbd-4fa7-9d83-5c8a3774f3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:54.098917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2152ad-9cbd-4fa7-9d83-5c8a3774f3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:54.118756 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0b2152ad-9cbd-4fa7-9d83-5c8a3774f3c0 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:54.119956 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0b2152ad-9cbd-4fa7-9d83-5c8a3774f3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:54.137497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:54.184372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:54.185976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:54.191553 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1134/2278] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:54.201532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2152ad-9cbd-4fa7-9d83-5c8a3774f3c0 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:54.203886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0b2152ad-9cbd-4fa7-9d83-5c8a3774f3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:54.207833 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feced99-f658-4cf2-adb5-2a0170e18794 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:54.213195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:54.214190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:54.214618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:54.215012 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] PATH_INFO: `/v3/projects/44f6aa04a3b04d9e9180e2c9654b6d09` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:54.217094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f6aa04a3b04d9e9180e2c9654b6d09)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:54.220585 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:54.276436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2152ad-9cbd-4fa7-9d83-5c8a3774f3c0 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:54.278790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f6aa04a3b04d9e9180e2c9654b6d09', 'request_id': 'req-0eb160e6-30af-4a24-908c-4aa219b08c16'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.285642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f6aa04a3b04d9e9180e2c9654b6d09', 'request_id': 'req-0eb160e6-30af-4a24-908c-4aa219b08c16'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.293654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f6aa04a3b04d9e9180e2c9654b6d09', 'request_id': 'req-0eb160e6-30af-4a24-908c-4aa219b08c16'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.315122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] The token cache is being invalidate because project 44f6aa04a3b04d9e9180e2c9654b6d09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:54.315475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.316330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.318303 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1145/2279] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:54 2022] DELETE /identity/v3/projects/44f6aa04a3b04d9e9180e2c9654b6d09 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:52:54.322020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:54.323447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:54.333279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a657b-10ce-4513-95b1-0dd7fdb8595a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:54.334000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a657b-10ce-4513-95b1-0dd7fdb8595a service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:54.334384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a657b-10ce-4513-95b1-0dd7fdb8595a service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:54.334668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a657b-10ce-4513-95b1-0dd7fdb8595a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:54.336448 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6e7a657b-10ce-4513-95b1-0dd7fdb8595a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:54.387408 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb160e6-30af-4a24-908c-4aa219b08c16 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:54.392195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:54.392822 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:54.393101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:54.393368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] PATH_INFO: `/v3/users/bafd51603b244943963777bc16090840` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:54.394589 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bafd51603b244943963777bc16090840)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:54.396602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:54.404371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a657b-10ce-4513-95b1-0dd7fdb8595a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:54.406692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a657b-10ce-4513-95b1-0dd7fdb8595a service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:54.417107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1135/2280] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:52:54.438774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:54.450180 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bafd51603b244943963777bc16090840'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.450993 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bafd51603b244943963777bc16090840'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.465458 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bafd51603b244943963777bc16090840'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.466576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bafd51603b244943963777bc16090840'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.476061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bafd51603b244943963777bc16090840'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.490716 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1146/2281] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:54 2022] DELETE /identity/v3/users/bafd51603b244943963777bc16090840 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:52:54.679598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a657b-10ce-4513-95b1-0dd7fdb8595a service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:54.690840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a657b-10ce-4513-95b1-0dd7fdb8595a service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:54.726530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:54.736335 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:54.737954 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:54.764012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:54.765218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:54.765608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:54.765878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] PATH_INFO: `/v3/projects/3349f60b943742cbb09403eea109fd07` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:54.767191 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3349f60b943742cbb09403eea109fd07)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:54.769390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:54.809377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921c6a7-5f86-44f8-8889-02da889e9fc9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:54.818189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3349f60b943742cbb09403eea109fd07', 'request_id': 'req-81446eb6-cc30-435e-9f4a-95a5089e7388'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.825965 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3349f60b943742cbb09403eea109fd07', 'request_id': 'req-81446eb6-cc30-435e-9f4a-95a5089e7388'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.833301 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3349f60b943742cbb09403eea109fd07', 'request_id': 'req-81446eb6-cc30-435e-9f4a-95a5089e7388'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.846484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] The token cache is being invalidate because project 3349f60b943742cbb09403eea109fd07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:54.846937 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.847781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-81446eb6-cc30-435e-9f4a-95a5089e7388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.849545 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1136/2282] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:54 2022] DELETE /identity/v3/projects/3349f60b943742cbb09403eea109fd07 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:54.854680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb9740-09e9-41b0-abda-d597c9df92ff None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:54.855027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb9740-09e9-41b0-abda-d597c9df92ff None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:54.855296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb9740-09e9-41b0-abda-d597c9df92ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:54.872660 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:54.873425 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:54.874340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:54.874616 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] PATH_INFO: `/v3/projects/b38e8307e3d94032845d8098b624bc80` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:54.875157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-03fb9740-09e9-41b0-abda-d597c9df92ff None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:54.875759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38e8307e3d94032845d8098b624bc80)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:54.876268 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-03fb9740-09e9-41b0-abda-d597c9df92ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:54.877655 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:54.884736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1137/2283] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:52:54.892965 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29e61e-f970-42db-85d8-26f6aaf93078 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:54.893533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29e61e-f970-42db-85d8-26f6aaf93078 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:54.893533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29e61e-f970-42db-85d8-26f6aaf93078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:54.914315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4a29e61e-f970-42db-85d8-26f6aaf93078 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:54.915517 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4a29e61e-f970-42db-85d8-26f6aaf93078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:54.927734 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38e8307e3d94032845d8098b624bc80', 'request_id': 'req-9c034eb5-6897-4f49-8024-349607481a06'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.933421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38e8307e3d94032845d8098b624bc80', 'request_id': 'req-9c034eb5-6897-4f49-8024-349607481a06'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.935935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:54.943488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38e8307e3d94032845d8098b624bc80', 'request_id': 'req-9c034eb5-6897-4f49-8024-349607481a06'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.956230 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] The token cache is being invalidate because project b38e8307e3d94032845d8098b624bc80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:54.956605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.957301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:54.959147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1147/2284] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:54 2022] DELETE /identity/v3/projects/b38e8307e3d94032845d8098b624bc80 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:52:54.962876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:54.964165 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:54.981190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1138/2285] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:52:54.985911 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29e61e-f970-42db-85d8-26f6aaf93078 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:54.999726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29e61e-f970-42db-85d8-26f6aaf93078 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.004906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3668c0-a49c-4478-aa31-09695805945e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.005622 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3668c0-a49c-4478-aa31-09695805945e admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.006027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3668c0-a49c-4478-aa31-09695805945e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.006432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3668c0-a49c-4478-aa31-09695805945e admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.007600 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3668c0-a49c-4478-aa31-09695805945e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:52:55.008006 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3668c0-a49c-4478-aa31-09695805945e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.010084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3668c0-a49c-4478-aa31-09695805945e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.023091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1139/2286] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:52:55.027189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3668c0-a49c-4478-aa31-09695805945e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.035322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c034eb5-6897-4f49-8024-349607481a06 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.037260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3668c0-a49c-4478-aa31-09695805945e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.039910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.040533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.040810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.041100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] PATH_INFO: `/v3/users/dbb23e955c1f46db867121dc4861abda` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.041910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de8c3-661e-4db8-aa76-b34b820c6cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.042227 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbb23e955c1f46db867121dc4861abda)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.042523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de8c3-661e-4db8-aa76-b34b820c6cac admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.042805 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de8c3-661e-4db8-aa76-b34b820c6cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.043079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de8c3-661e-4db8-aa76-b34b820c6cac admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.043705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37de8c3-661e-4db8-aa76-b34b820c6cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.044124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.045621 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37de8c3-661e-4db8-aa76-b34b820c6cac admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.073644 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1140/2287] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:55 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:52:55.078850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de8c3-661e-4db8-aa76-b34b820c6cac admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.096219 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbb23e955c1f46db867121dc4861abda'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:55.096895 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbb23e955c1f46db867121dc4861abda'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:55.099070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:55.109713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbb23e955c1f46db867121dc4861abda'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:55.110734 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbb23e955c1f46db867121dc4861abda'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:55.118412 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbb23e955c1f46db867121dc4861abda'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:55.132607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1148/2288] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:52:54 2022] DELETE /identity/v3/users/dbb23e955c1f46db867121dc4861abda => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:52:55.147461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de8c3-661e-4db8-aa76-b34b820c6cac admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.214792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b457b0a-9b9f-40e9-8c24-106feb8eac34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.215477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b457b0a-9b9f-40e9-8c24-106feb8eac34 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.215728 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b457b0a-9b9f-40e9-8c24-106feb8eac34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.215974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b457b0a-9b9f-40e9-8c24-106feb8eac34 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.216620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b457b0a-9b9f-40e9-8c24-106feb8eac34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.219107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b457b0a-9b9f-40e9-8c24-106feb8eac34 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.242046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1141/2289] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:55 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:55.246549 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.248021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:55.261579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25b34a-2138-4d95-85e1-ddd227c1a4c3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.266936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17c65-dcd7-49ea-b2db-1e9a7f22979f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.267539 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17c65-dcd7-49ea-b2db-1e9a7f22979f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.267818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17c65-dcd7-49ea-b2db-1e9a7f22979f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.268093 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17c65-dcd7-49ea-b2db-1e9a7f22979f admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.268728 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f17c65-dcd7-49ea-b2db-1e9a7f22979f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.270348 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f17c65-dcd7-49ea-b2db-1e9a7f22979f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.280789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1149/2290] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:55.284920 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b457b0a-9b9f-40e9-8c24-106feb8eac34 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.297034 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b457b0a-9b9f-40e9-8c24-106feb8eac34 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.308012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3e74c-7575-4d01-8ea5-eb12cfcef710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.308628 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3e74c-7575-4d01-8ea5-eb12cfcef710 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.308912 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3e74c-7575-4d01-8ea5-eb12cfcef710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.309188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3e74c-7575-4d01-8ea5-eb12cfcef710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.309840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3e74c-7575-4d01-8ea5-eb12cfcef710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.312168 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3e74c-7575-4d01-8ea5-eb12cfcef710 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.322874 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1142/2291] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:55.327173 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17c65-dcd7-49ea-b2db-1e9a7f22979f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.337159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17c65-dcd7-49ea-b2db-1e9a7f22979f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.342289 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb966b0-e2c4-4ab2-b52b-7ffe0a85e272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.343014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb966b0-e2c4-4ab2-b52b-7ffe0a85e272 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.343309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb966b0-e2c4-4ab2-b52b-7ffe0a85e272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.343566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb966b0-e2c4-4ab2-b52b-7ffe0a85e272 admin admin] PATH_INFO: `/v3/projects/9c61c491e2924ffe95acc3f4e189d37d/users/0d46e4243350431b9ff792b269b9e4be/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.358908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb966b0-e2c4-4ab2-b52b-7ffe0a85e272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c61c491e2924ffe95acc3f4e189d37d, user_id=0d46e4243350431b9ff792b269b9e4be, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.361584 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb966b0-e2c4-4ab2-b52b-7ffe0a85e272 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.375885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1150/2292] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:55 2022] PUT /identity/v3/projects/9c61c491e2924ffe95acc3f4e189d37d/users/0d46e4243350431b9ff792b269b9e4be/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:55.427033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3e74c-7575-4d01-8ea5-eb12cfcef710 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.428689 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4ec3e74c-7575-4d01-8ea5-eb12cfcef710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:55.445032 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea920d7-50d0-4d71-a673-b31c8343e60b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.445322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea920d7-50d0-4d71-a673-b31c8343e60b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.446217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea920d7-50d0-4d71-a673-b31c8343e60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.451421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3e74c-7575-4d01-8ea5-eb12cfcef710 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.474414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-eea920d7-50d0-4d71-a673-b31c8343e60b None None] MFA Rules not processed for user `0d46e4243350431b9ff792b269b9e4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:55.475555 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-eea920d7-50d0-4d71-a673-b31c8343e60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:55.498298 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:55.510434 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.511091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.511368 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.511648 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] PATH_INFO: `/v3/projects/b5ef571941ab4d5ebd00c4b47bf86f30` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.513440 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ef571941ab4d5ebd00c4b47bf86f30)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.515589 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.549980 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1151/2293] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:52:55.562432 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea920d7-50d0-4d71-a673-b31c8343e60b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.563676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:55.563825 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-eea920d7-50d0-4d71-a673-b31c8343e60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:55.567262 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ef571941ab4d5ebd00c4b47bf86f30', 'request_id': 'req-e4065e7f-8a4d-4428-9c55-b545d33abdcb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:55.576372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ef571941ab4d5ebd00c4b47bf86f30', 'request_id': 'req-e4065e7f-8a4d-4428-9c55-b545d33abdcb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:55.584013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ef571941ab4d5ebd00c4b47bf86f30', 'request_id': 'req-e4065e7f-8a4d-4428-9c55-b545d33abdcb'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:55.595302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] The token cache is being invalidate because project b5ef571941ab4d5ebd00c4b47bf86f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:52:55.595630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:55.596463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e4065e7f-8a4d-4428-9c55-b545d33abdcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:52:55.597998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1143/2294] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:52:55 2022] DELETE /identity/v3/projects/b5ef571941ab4d5ebd00c4b47bf86f30 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:55.609256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb20707-073f-4dc7-82cc-1ee18a01cb84 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.609899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb20707-073f-4dc7-82cc-1ee18a01cb84 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.610208 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb20707-073f-4dc7-82cc-1ee18a01cb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.631394 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea920d7-50d0-4d71-a673-b31c8343e60b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.631638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-aeb20707-073f-4dc7-82cc-1ee18a01cb84 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:55.632583 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-aeb20707-073f-4dc7-82cc-1ee18a01cb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:55.681047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d48a0d-0ad3-4734-b83f-ddf711de7c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.681650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d48a0d-0ad3-4734-b83f-ddf711de7c2e service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.681937 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d48a0d-0ad3-4734-b83f-ddf711de7c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.682235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d48a0d-0ad3-4734-b83f-ddf711de7c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.683978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-68d48a0d-0ad3-4734-b83f-ddf711de7c2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:55.693374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1144/2295] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:55.701177 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3f422-79ba-4c97-9167-5ac6f334597d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.701455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3f422-79ba-4c97-9167-5ac6f334597d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.701759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3f422-79ba-4c97-9167-5ac6f334597d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.723248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-16c3f422-79ba-4c97-9167-5ac6f334597d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:55.724943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-16c3f422-79ba-4c97-9167-5ac6f334597d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:55.734041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1145/2296] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:52:55.738232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3f422-79ba-4c97-9167-5ac6f334597d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.744534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d48a0d-0ad3-4734-b83f-ddf711de7c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.745964 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d48a0d-0ad3-4734-b83f-ddf711de7c2e service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.747227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3f422-79ba-4c97-9167-5ac6f334597d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.752129 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77716d-bb46-4d07-814d-feb547a7e00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.752730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77716d-bb46-4d07-814d-feb547a7e00d admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.752995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77716d-bb46-4d07-814d-feb547a7e00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.753265 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77716d-bb46-4d07-814d-feb547a7e00d admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.753906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77716d-bb46-4d07-814d-feb547a7e00d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:52:55.754241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77716d-bb46-4d07-814d-feb547a7e00d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.756109 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b77716d-bb46-4d07-814d-feb547a7e00d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.756243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1152/2297] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:52:55.771882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1146/2298] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:52:55.773020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d48a0d-0ad3-4734-b83f-ddf711de7c2e service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.784845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d48a0d-0ad3-4734-b83f-ddf711de7c2e service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.789383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c2437-4d0e-4efb-885c-596210032ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.789970 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c2437-4d0e-4efb-885c-596210032ffd admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.790291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c2437-4d0e-4efb-885c-596210032ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.790559 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c2437-4d0e-4efb-885c-596210032ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.791172 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c2437-4d0e-4efb-885c-596210032ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.793041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c2437-4d0e-4efb-885c-596210032ffd admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.825344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1153/2299] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:55 2022] POST /identity/v3/projects => generated 409 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:52:55.827202 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77716d-bb46-4d07-814d-feb547a7e00d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.837285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77716d-bb46-4d07-814d-feb547a7e00d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.894295 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1706f7f-c165-41c3-9739-6e39d4fbf92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.895140 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1706f7f-c165-41c3-9739-6e39d4fbf92c admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.895325 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1706f7f-c165-41c3-9739-6e39d4fbf92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.895663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1706f7f-c165-41c3-9739-6e39d4fbf92c admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.896294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1706f7f-c165-41c3-9739-6e39d4fbf92c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.898201 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1706f7f-c165-41c3-9739-6e39d4fbf92c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.925643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1147/2300] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:55 2022] POST /identity/v3/users => generated 305 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:55.926870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c2437-4d0e-4efb-885c-596210032ffd admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.938410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c2437-4d0e-4efb-885c-596210032ffd admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.943742 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6163d-96a8-4322-a2fc-68bc7cc54ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.944346 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6163d-96a8-4322-a2fc-68bc7cc54ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.944609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6163d-96a8-4322-a2fc-68bc7cc54ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.944875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6163d-96a8-4322-a2fc-68bc7cc54ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.945507 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6163d-96a8-4322-a2fc-68bc7cc54ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.947115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6163d-96a8-4322-a2fc-68bc7cc54ad8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.953825 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:55.963219 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1154/2301] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:55.964436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1706f7f-c165-41c3-9739-6e39d4fbf92c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:55.975863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1706f7f-c165-41c3-9739-6e39d4fbf92c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:55.980573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c10ac-d484-4e66-93d9-5a469c94fe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:55.981270 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c10ac-d484-4e66-93d9-5a469c94fe2a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:55.981540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c10ac-d484-4e66-93d9-5a469c94fe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:55.981804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c10ac-d484-4e66-93d9-5a469c94fe2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:55.982473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c10ac-d484-4e66-93d9-5a469c94fe2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:55.984711 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c10ac-d484-4e66-93d9-5a469c94fe2a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:55.999892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1148/2302] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:56.007134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6163d-96a8-4322-a2fc-68bc7cc54ad8 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:56.018341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6163d-96a8-4322-a2fc-68bc7cc54ad8 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:56.023295 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9575ae-9e45-4bd8-991a-65ca72d59e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:56.023950 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9575ae-9e45-4bd8-991a-65ca72d59e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:56.024238 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9575ae-9e45-4bd8-991a-65ca72d59e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:56.024510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9575ae-9e45-4bd8-991a-65ca72d59e80 admin admin] PATH_INFO: `/v3/projects/90e0d6428ef545829e29ac99c88b9d50/users/1fa92a3887a34f20a9ecd453ad00cc57/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:56.040748 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9575ae-9e45-4bd8-991a-65ca72d59e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e0d6428ef545829e29ac99c88b9d50, user_id=1fa92a3887a34f20a9ecd453ad00cc57, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:56.043446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9575ae-9e45-4bd8-991a-65ca72d59e80 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:56.062238 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1155/2303] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:56 2022] PUT /identity/v3/projects/90e0d6428ef545829e29ac99c88b9d50/users/1fa92a3887a34f20a9ecd453ad00cc57/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:56.069206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dc9db-409b-4fb7-9d55-032eb2be8c59 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:56.069504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dc9db-409b-4fb7-9d55-032eb2be8c59 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:56.077842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dc9db-409b-4fb7-9d55-032eb2be8c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:56.100429 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-cb0dc9db-409b-4fb7-9d55-032eb2be8c59 None None] MFA Rules not processed for user `1fa92a3887a34f20a9ecd453ad00cc57`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:56.101686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-cb0dc9db-409b-4fb7-9d55-032eb2be8c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:56.122115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:56.169361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1149/2304] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:56 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:56.174121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9575ae-9e45-4bd8-991a-65ca72d59e80 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:56.236870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9575ae-9e45-4bd8-991a-65ca72d59e80 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:56.241449 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a6df5-c8ee-4778-baa1-6c6570df7363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:56.242050 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a6df5-c8ee-4778-baa1-6c6570df7363 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:56.242375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a6df5-c8ee-4778-baa1-6c6570df7363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:56.242650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a6df5-c8ee-4778-baa1-6c6570df7363 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:56.243299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a6df5-c8ee-4778-baa1-6c6570df7363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:56.245175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a6df5-c8ee-4778-baa1-6c6570df7363 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:56.271412 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1156/2305] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:52:56 2022] POST /identity/v3/projects => generated 409 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:56.275698 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dc9db-409b-4fb7-9d55-032eb2be8c59 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:56.287098 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dc9db-409b-4fb7-9d55-032eb2be8c59 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:56.342080 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db090ca-caf7-4165-9316-097c55ec6ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:56.343315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db090ca-caf7-4165-9316-097c55ec6ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:56.343676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db090ca-caf7-4165-9316-097c55ec6ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:56.343948 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db090ca-caf7-4165-9316-097c55ec6ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:56.344578 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db090ca-caf7-4165-9316-097c55ec6ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:56.346579 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db090ca-caf7-4165-9316-097c55ec6ec3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:56.367332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1150/2306] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:52:56 2022] POST /identity/v3/users => generated 311 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:52:56.371557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a6df5-c8ee-4778-baa1-6c6570df7363 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:56.382876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a6df5-c8ee-4778-baa1-6c6570df7363 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:56.387449 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016b361-6add-4963-a080-dc5774f7b459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:56.388038 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016b361-6add-4963-a080-dc5774f7b459 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:56.388316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016b361-6add-4963-a080-dc5774f7b459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:56.388587 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016b361-6add-4963-a080-dc5774f7b459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:56.389219 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016b361-6add-4963-a080-dc5774f7b459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:56.391416 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016b361-6add-4963-a080-dc5774f7b459 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:56.401936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1157/2307] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:52:56.406597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db090ca-caf7-4165-9316-097c55ec6ec3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:56.416530 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db090ca-caf7-4165-9316-097c55ec6ec3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:56.421184 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7ec63-0fa6-40ae-804d-27eb75f05dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:56.421821 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7ec63-0fa6-40ae-804d-27eb75f05dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:56.422103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7ec63-0fa6-40ae-804d-27eb75f05dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:56.422410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7ec63-0fa6-40ae-804d-27eb75f05dbc admin admin] PATH_INFO: `/v3/projects/f6581591d7934b34aed3a27f617fbf01/users/6741275bcbcc45bb969a73eb6001f0a8/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:56.437668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7ec63-0fa6-40ae-804d-27eb75f05dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6581591d7934b34aed3a27f617fbf01, user_id=6741275bcbcc45bb969a73eb6001f0a8, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:56.440390 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7ec63-0fa6-40ae-804d-27eb75f05dbc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:56.453840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1151/2308] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:56 2022] PUT /identity/v3/projects/f6581591d7934b34aed3a27f617fbf01/users/6741275bcbcc45bb969a73eb6001f0a8/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:56.457764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016b361-6add-4963-a080-dc5774f7b459 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:56.468157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016b361-6add-4963-a080-dc5774f7b459 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:56.526393 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59a324-8fac-4596-b10f-db2f9490d6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:56.526936 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59a324-8fac-4596-b10f-db2f9490d6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:56.527213 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59a324-8fac-4596-b10f-db2f9490d6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:56.527477 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59a324-8fac-4596-b10f-db2f9490d6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:56.528125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59a324-8fac-4596-b10f-db2f9490d6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:56.529691 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59a324-8fac-4596-b10f-db2f9490d6b4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:56.539870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1158/2309] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:52:56.543986 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7ec63-0fa6-40ae-804d-27eb75f05dbc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:56.554408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7ec63-0fa6-40ae-804d-27eb75f05dbc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:56.559110 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e12d34-1b0b-4294-9af0-37baf3254d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:56.559780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e12d34-1b0b-4294-9af0-37baf3254d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:56.560111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e12d34-1b0b-4294-9af0-37baf3254d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:56.560388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e12d34-1b0b-4294-9af0-37baf3254d65 admin admin] PATH_INFO: `/v3/projects/f6581591d7934b34aed3a27f617fbf01/users/6741275bcbcc45bb969a73eb6001f0a8/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:56.563027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e12d34-1b0b-4294-9af0-37baf3254d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6581591d7934b34aed3a27f617fbf01, user_id=6741275bcbcc45bb969a73eb6001f0a8, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:56.565679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e12d34-1b0b-4294-9af0-37baf3254d65 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:56.575785 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1152/2310] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:52:56 2022] PUT /identity/v3/projects/f6581591d7934b34aed3a27f617fbf01/users/6741275bcbcc45bb969a73eb6001f0a8/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:52:56.586622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a3c4d-afa7-4132-af78-6eef11cac972 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:56.586904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a3c4d-afa7-4132-af78-6eef11cac972 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:56.587172 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a3c4d-afa7-4132-af78-6eef11cac972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:56.615179 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-864a3c4d-afa7-4132-af78-6eef11cac972 None None] MFA Rules not processed for user `6741275bcbcc45bb969a73eb6001f0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:56.616355 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-864a3c4d-afa7-4132-af78-6eef11cac972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:56.654577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:56.690580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1159/2311] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:56 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:56.701514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e12d34-1b0b-4294-9af0-37baf3254d65 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:56.703842 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-70e12d34-1b0b-4294-9af0-37baf3254d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:56.760834 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e12d34-1b0b-4294-9af0-37baf3254d65 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:56.765011 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524810b-d957-4592-988d-9697ffd78fd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:56.765663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524810b-d957-4592-988d-9697ffd78fd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:56.765936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524810b-d957-4592-988d-9697ffd78fd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:56.766231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524810b-d957-4592-988d-9697ffd78fd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:56.774981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524810b-d957-4592-988d-9697ffd78fd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:56.776529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524810b-d957-4592-988d-9697ffd78fd2 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:56.785783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1153/2312] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:56 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:56.946593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a3c4d-afa7-4132-af78-6eef11cac972 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:56.956910 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a3c4d-afa7-4132-af78-6eef11cac972 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:56.960610 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b471b-108b-4a20-86ed-478a7f970b26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:56.961240 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b471b-108b-4a20-86ed-478a7f970b26 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:56.961522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b471b-108b-4a20-86ed-478a7f970b26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:56.961793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b471b-108b-4a20-86ed-478a7f970b26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:57.026124 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b471b-108b-4a20-86ed-478a7f970b26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:57.027636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b471b-108b-4a20-86ed-478a7f970b26 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:57.036858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1160/2313] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:56 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:57.149798 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa47d55-ed48-4f78-9582-9a6723074e2e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:57.150565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa47d55-ed48-4f78-9582-9a6723074e2e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:57.150841 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa47d55-ed48-4f78-9582-9a6723074e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:57.172643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-aaa47d55-ed48-4f78-9582-9a6723074e2e None None] MFA Rules not processed for user `0d46e4243350431b9ff792b269b9e4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:52:57.173951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-aaa47d55-ed48-4f78-9582-9a6723074e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:57.186950 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:52:57.240213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1154/2314] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:52:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:52:57.715565 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b471b-108b-4a20-86ed-478a7f970b26 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:57.716964 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e03b471b-108b-4a20-86ed-478a7f970b26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:57.788767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b471b-108b-4a20-86ed-478a7f970b26 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:57.793952 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38002dd0-d9e1-429f-bf85-b605c4dd9695 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:57.794583 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38002dd0-d9e1-429f-bf85-b605c4dd9695 service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:57.794864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38002dd0-d9e1-429f-bf85-b605c4dd9695 service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:57.795136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38002dd0-d9e1-429f-bf85-b605c4dd9695 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:57.797460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-38002dd0-d9e1-429f-bf85-b605c4dd9695 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:57.808892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38002dd0-d9e1-429f-bf85-b605c4dd9695 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:57.810425 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38002dd0-d9e1-429f-bf85-b605c4dd9695 service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:57.823235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1161/2315] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:52:58.266725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa47d55-ed48-4f78-9582-9a6723074e2e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:52:58.268590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-aaa47d55-ed48-4f78-9582-9a6723074e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:58.330629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa47d55-ed48-4f78-9582-9a6723074e2e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:52:58.334014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730e760-fe4d-482c-b48c-0d6dffe47bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:52:58.335278 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730e760-fe4d-482c-b48c-0d6dffe47bdf service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:52:58.335532 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730e760-fe4d-482c-b48c-0d6dffe47bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:52:58.335796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730e760-fe4d-482c-b48c-0d6dffe47bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:52:58.337550 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0730e760-fe4d-482c-b48c-0d6dffe47bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:52:58.349858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730e760-fe4d-482c-b48c-0d6dffe47bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:52:58.351299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730e760-fe4d-482c-b48c-0d6dffe47bdf service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:52:58.361770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1155/2316] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:52:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:53:05.521252 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38002dd0-d9e1-429f-bf85-b605c4dd9695 service placement] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:05.583069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38002dd0-d9e1-429f-bf85-b605c4dd9695 service placement] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:05.587592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:05.588239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:05.588520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:05.588797 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] PATH_INFO: `/v3/users/1fa92a3887a34f20a9ecd453ad00cc57` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:05.589897 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fa92a3887a34f20a9ecd453ad00cc57)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:05.591843 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:05.594326 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:05.640532 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fa92a3887a34f20a9ecd453ad00cc57'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:05.641100 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fa92a3887a34f20a9ecd453ad00cc57'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:05.653031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fa92a3887a34f20a9ecd453ad00cc57'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:05.653658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fa92a3887a34f20a9ecd453ad00cc57'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:05.661466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fa92a3887a34f20a9ecd453ad00cc57'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:05.674517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1162/2317] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:05 2022] DELETE /identity/v3/users/1fa92a3887a34f20a9ecd453ad00cc57 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:05.679019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354c45b-c1a6-4915-a2ae-2d2d4e6e66e8 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:05.679513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354c45b-c1a6-4915-a2ae-2d2d4e6e66e8 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:05.679614 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354c45b-c1a6-4915-a2ae-2d2d4e6e66e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:05.698130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-0354c45b-c1a6-4915-a2ae-2d2d4e6e66e8 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:05.699339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0354c45b-c1a6-4915-a2ae-2d2d4e6e66e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:05.756544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1156/2318] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:05.766972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:05.768375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:05.825406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4a320-1495-4a22-b4ef-6193b227ec32 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:05.830018 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e489b3a-3093-4741-aadb-9fc88a63e2a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:05.830738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e489b3a-3093-4741-aadb-9fc88a63e2a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:05.830987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e489b3a-3093-4741-aadb-9fc88a63e2a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:05.831260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e489b3a-3093-4741-aadb-9fc88a63e2a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:05.847838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e489b3a-3093-4741-aadb-9fc88a63e2a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:05.849274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e489b3a-3093-4741-aadb-9fc88a63e2a1 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:05.860323 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1163/2319] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:06.129884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354c45b-c1a6-4915-a2ae-2d2d4e6e66e8 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:06.131378 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-0354c45b-c1a6-4915-a2ae-2d2d4e6e66e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:06.145588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354c45b-c1a6-4915-a2ae-2d2d4e6e66e8 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:06.150291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:06.150925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:06.151204 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:06.151479 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] PATH_INFO: `/v3/projects/90e0d6428ef545829e29ac99c88b9d50` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:06.152616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e0d6428ef545829e29ac99c88b9d50)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:06.154558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:06.201668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e0d6428ef545829e29ac99c88b9d50', 'request_id': 'req-7a6e2fa8-d29f-472d-a30e-40d98acdef50'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.210544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e0d6428ef545829e29ac99c88b9d50', 'request_id': 'req-7a6e2fa8-d29f-472d-a30e-40d98acdef50'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.216887 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e0d6428ef545829e29ac99c88b9d50', 'request_id': 'req-7a6e2fa8-d29f-472d-a30e-40d98acdef50'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.218139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:06.229993 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] The token cache is being invalidate because project 90e0d6428ef545829e29ac99c88b9d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:06.230307 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.230942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.232415 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1157/2320] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:06 2022] DELETE /identity/v3/projects/90e0d6428ef545829e29ac99c88b9d50 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:06.237015 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e489b3a-3093-4741-aadb-9fc88a63e2a1 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:06.246366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e489b3a-3093-4741-aadb-9fc88a63e2a1 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:06.303293 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bcbc1-e5bc-458d-8d44-ab98961d6e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:06.303914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bcbc1-e5bc-458d-8d44-ab98961d6e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:06.304188 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bcbc1-e5bc-458d-8d44-ab98961d6e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:06.304476 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bcbc1-e5bc-458d-8d44-ab98961d6e91 admin admin] PATH_INFO: `/v3/users/6741275bcbcc45bb969a73eb6001f0a8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:06.305598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bcbc1-e5bc-458d-8d44-ab98961d6e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6741275bcbcc45bb969a73eb6001f0a8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:06.307573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bcbc1-e5bc-458d-8d44-ab98961d6e91 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:06.333344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:06.359270 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7f5bcbc1-e5bc-458d-8d44-ab98961d6e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6741275bcbcc45bb969a73eb6001f0a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.359928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7f5bcbc1-e5bc-458d-8d44-ab98961d6e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6741275bcbcc45bb969a73eb6001f0a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.372364 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7f5bcbc1-e5bc-458d-8d44-ab98961d6e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6741275bcbcc45bb969a73eb6001f0a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.373007 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7f5bcbc1-e5bc-458d-8d44-ab98961d6e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6741275bcbcc45bb969a73eb6001f0a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.381143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7f5bcbc1-e5bc-458d-8d44-ab98961d6e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6741275bcbcc45bb969a73eb6001f0a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.395073 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1164/2321] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:06 2022] DELETE /identity/v3/users/6741275bcbcc45bb969a73eb6001f0a8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:06.808726 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:06.810441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:06.835086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e2fa8-d29f-472d-a30e-40d98acdef50 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:06.899172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:06.899829 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:06.900104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:06.900371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] PATH_INFO: `/v3/projects/f6581591d7934b34aed3a27f617fbf01` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:06.901459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6581591d7934b34aed3a27f617fbf01)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:06.903452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:06.949869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6581591d7934b34aed3a27f617fbf01', 'request_id': 'req-2cf1f42e-be3d-4997-a970-c40b6ac09d45'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.957449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6581591d7934b34aed3a27f617fbf01', 'request_id': 'req-2cf1f42e-be3d-4997-a970-c40b6ac09d45'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.963588 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6581591d7934b34aed3a27f617fbf01', 'request_id': 'req-2cf1f42e-be3d-4997-a970-c40b6ac09d45'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.975573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] The token cache is being invalidate because project f6581591d7934b34aed3a27f617fbf01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:06.975856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.976492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-2cf1f42e-be3d-4997-a970-c40b6ac09d45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:06.977975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1158/2322] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:06 2022] DELETE /identity/v3/projects/f6581591d7934b34aed3a27f617fbf01 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:06.983044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99be404-0267-4617-853e-14482e37ed41 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:06.983500 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99be404-0267-4617-853e-14482e37ed41 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:06.983500 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99be404-0267-4617-853e-14482e37ed41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.003941 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-a99be404-0267-4617-853e-14482e37ed41 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:07.005076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a99be404-0267-4617-853e-14482e37ed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:07.069434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1165/2323] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:07.078549 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e24b4-9a5e-4569-9ec7-a98435f536dc None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.078850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e24b4-9a5e-4569-9ec7-a98435f536dc None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.079111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e24b4-9a5e-4569-9ec7-a98435f536dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.099319 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b84e24b4-9a5e-4569-9ec7-a98435f536dc None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:07.100523 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b84e24b4-9a5e-4569-9ec7-a98435f536dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:07.108863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1159/2324] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:07.113545 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99be404-0267-4617-853e-14482e37ed41 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:07.123040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99be404-0267-4617-853e-14482e37ed41 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:07.127932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae70604-a312-42c4-84ce-4fc35646b2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:07.128535 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae70604-a312-42c4-84ce-4fc35646b2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.128816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae70604-a312-42c4-84ce-4fc35646b2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.129088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae70604-a312-42c4-84ce-4fc35646b2f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.129733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae70604-a312-42c4-84ce-4fc35646b2f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:53:07.130036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae70604-a312-42c4-84ce-4fc35646b2f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:07.131685 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae70604-a312-42c4-84ce-4fc35646b2f2 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:07.144143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1166/2325] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:53:07.148443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e24b4-9a5e-4569-9ec7-a98435f536dc None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:07.158765 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e24b4-9a5e-4569-9ec7-a98435f536dc None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:07.163351 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a53609-b950-4397-84f5-5f02ace5f01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:07.163942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a53609-b950-4397-84f5-5f02ace5f01e admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.164216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a53609-b950-4397-84f5-5f02ace5f01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.164484 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a53609-b950-4397-84f5-5f02ace5f01e admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.165106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a53609-b950-4397-84f5-5f02ace5f01e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:07.167076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a53609-b950-4397-84f5-5f02ace5f01e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:07.194846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1160/2326] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:07 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:07.199170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae70604-a312-42c4-84ce-4fc35646b2f2 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:07.209294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae70604-a312-42c4-84ce-4fc35646b2f2 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:07.267887 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bb78a-9573-4899-a915-a88e05b3e571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:07.268566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb78a-9573-4899-a915-a88e05b3e571 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.268846 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb78a-9573-4899-a915-a88e05b3e571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.269116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb78a-9573-4899-a915-a88e05b3e571 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.269780 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bb78a-9573-4899-a915-a88e05b3e571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:07.271770 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bb78a-9573-4899-a915-a88e05b3e571 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:07.292839 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1167/2327] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:07 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:07.297438 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a53609-b950-4397-84f5-5f02ace5f01e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:07.307663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a53609-b950-4397-84f5-5f02ace5f01e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:07.312966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff2313-acb7-4705-bb77-6009ec15dc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:07.313557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff2313-acb7-4705-bb77-6009ec15dc31 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.313837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff2313-acb7-4705-bb77-6009ec15dc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.314109 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff2313-acb7-4705-bb77-6009ec15dc31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.314780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff2313-acb7-4705-bb77-6009ec15dc31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:07.316369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff2313-acb7-4705-bb77-6009ec15dc31 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:07.326112 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1161/2328] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:07.330241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bb78a-9573-4899-a915-a88e05b3e571 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:07.339895 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bb78a-9573-4899-a915-a88e05b3e571 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:07.344468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea4830-b7b5-4e2b-a6e2-c602eaa1cae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:07.345046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea4830-b7b5-4e2b-a6e2-c602eaa1cae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.345321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea4830-b7b5-4e2b-a6e2-c602eaa1cae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.345586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea4830-b7b5-4e2b-a6e2-c602eaa1cae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.346246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea4830-b7b5-4e2b-a6e2-c602eaa1cae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:07.347811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea4830-b7b5-4e2b-a6e2-c602eaa1cae8 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:07.358232 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1168/2329] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:07.362829 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff2313-acb7-4705-bb77-6009ec15dc31 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:07.372757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff2313-acb7-4705-bb77-6009ec15dc31 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:07.378080 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cecdd-598f-49d3-b9bf-f95f7b137c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:07.378743 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cecdd-598f-49d3-b9bf-f95f7b137c51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.379017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cecdd-598f-49d3-b9bf-f95f7b137c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.379284 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cecdd-598f-49d3-b9bf-f95f7b137c51 admin admin] PATH_INFO: `/v3/projects/29b0591db51e4ee5b2321c806614a577/users/b233a73f0f514429a637ff877cf87a7c/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.393723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028cecdd-598f-49d3-b9bf-f95f7b137c51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29b0591db51e4ee5b2321c806614a577, user_id=b233a73f0f514429a637ff877cf87a7c, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:07.396356 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028cecdd-598f-49d3-b9bf-f95f7b137c51 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:07.409868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1162/2330] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:07 2022] PUT /identity/v3/projects/29b0591db51e4ee5b2321c806614a577/users/b233a73f0f514429a637ff877cf87a7c/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:53:07.420077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82053-f5cd-48b3-8f8b-f2ffb5687b6c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.420386 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82053-f5cd-48b3-8f8b-f2ffb5687b6c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.420657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82053-f5cd-48b3-8f8b-f2ffb5687b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.434318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:07.450129 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-2eb82053-f5cd-48b3-8f8b-f2ffb5687b6c None None] MFA Rules not processed for user `b233a73f0f514429a637ff877cf87a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:07.451351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2eb82053-f5cd-48b3-8f8b-f2ffb5687b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:07.514409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1169/2331] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:07.519116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cecdd-598f-49d3-b9bf-f95f7b137c51 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:07.536466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:07.584082 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cecdd-598f-49d3-b9bf-f95f7b137c51 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:07.588649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669ab69-cbf1-4464-9e66-caa11035bd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:07.589322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669ab69-cbf1-4464-9e66-caa11035bd74 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.589609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669ab69-cbf1-4464-9e66-caa11035bd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.589875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669ab69-cbf1-4464-9e66-caa11035bd74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.590529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669ab69-cbf1-4464-9e66-caa11035bd74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:07.592405 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669ab69-cbf1-4464-9e66-caa11035bd74 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:07.618358 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1163/2332] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:07 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:07.622505 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb82053-f5cd-48b3-8f8b-f2ffb5687b6c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:07.633455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb82053-f5cd-48b3-8f8b-f2ffb5687b6c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:07.687858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606012f-72a0-4648-87ae-29c6d843353b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:07.688455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606012f-72a0-4648-87ae-29c6d843353b admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.688729 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606012f-72a0-4648-87ae-29c6d843353b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.688995 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606012f-72a0-4648-87ae-29c6d843353b admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.689615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606012f-72a0-4648-87ae-29c6d843353b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:07.691559 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606012f-72a0-4648-87ae-29c6d843353b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:07.712405 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1170/2333] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:07 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:07.716351 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669ab69-cbf1-4464-9e66-caa11035bd74 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:07.726438 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669ab69-cbf1-4464-9e66-caa11035bd74 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:07.730902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7517ba0-6972-4f63-98fd-4548c9458ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:07.731492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7517ba0-6972-4f63-98fd-4548c9458ddb admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.731769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7517ba0-6972-4f63-98fd-4548c9458ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.732036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7517ba0-6972-4f63-98fd-4548c9458ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.732662 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7517ba0-6972-4f63-98fd-4548c9458ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:07.734292 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7517ba0-6972-4f63-98fd-4548c9458ddb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:07.744510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1164/2334] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:53:07.748382 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606012f-72a0-4648-87ae-29c6d843353b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:07.758884 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606012f-72a0-4648-87ae-29c6d843353b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:07.763487 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8dcca-f0c4-43bc-940d-1f9d4d72cd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:07.764073 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8dcca-f0c4-43bc-940d-1f9d4d72cd21 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.764350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8dcca-f0c4-43bc-940d-1f9d4d72cd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.764620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8dcca-f0c4-43bc-940d-1f9d4d72cd21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.766057 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8dcca-f0c4-43bc-940d-1f9d4d72cd21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:07.767735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8dcca-f0c4-43bc-940d-1f9d4d72cd21 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:07.777534 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1171/2335] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:07.782716 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7517ba0-6972-4f63-98fd-4548c9458ddb admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:07.792455 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7517ba0-6972-4f63-98fd-4548c9458ddb admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:07.797718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9278baa-aeb7-4bfb-aaf2-eaa010352a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:07.798391 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278baa-aeb7-4bfb-aaf2-eaa010352a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.798676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278baa-aeb7-4bfb-aaf2-eaa010352a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.798945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278baa-aeb7-4bfb-aaf2-eaa010352a3c admin admin] PATH_INFO: `/v3/projects/0d077c96b2604548a50bf9d12b17cd88/users/7d826072fdfb4222bf2127c1f8af785e/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.813133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9278baa-aeb7-4bfb-aaf2-eaa010352a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d077c96b2604548a50bf9d12b17cd88, user_id=7d826072fdfb4222bf2127c1f8af785e, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:07.815810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9278baa-aeb7-4bfb-aaf2-eaa010352a3c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:07.828158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1165/2336] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:07 2022] PUT /identity/v3/projects/0d077c96b2604548a50bf9d12b17cd88/users/7d826072fdfb4222bf2127c1f8af785e/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:53:07.838536 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069d29c-84c1-42a4-83f5-8a999ab772d9 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:07.838826 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069d29c-84c1-42a4-83f5-8a999ab772d9 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:07.839098 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069d29c-84c1-42a4-83f5-8a999ab772d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:07.867634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9069d29c-84c1-42a4-83f5-8a999ab772d9 None None] MFA Rules not processed for user `7d826072fdfb4222bf2127c1f8af785e`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:07.868830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9069d29c-84c1-42a4-83f5-8a999ab772d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:07.934370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1172/2337] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:07.938977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9278baa-aeb7-4bfb-aaf2-eaa010352a3c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:08.000608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9278baa-aeb7-4bfb-aaf2-eaa010352a3c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:08.005257 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d98064-bd72-40d3-aff8-2e22131b92de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:08.005850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d98064-bd72-40d3-aff8-2e22131b92de admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:08.006128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d98064-bd72-40d3-aff8-2e22131b92de admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:08.006434 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d98064-bd72-40d3-aff8-2e22131b92de admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:08.007070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d98064-bd72-40d3-aff8-2e22131b92de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:08.008945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d98064-bd72-40d3-aff8-2e22131b92de admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:08.036354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1166/2338] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:07 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:08.040391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069d29c-84c1-42a4-83f5-8a999ab772d9 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:08.047159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:08.051634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069d29c-84c1-42a4-83f5-8a999ab772d9 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:08.108296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47cb312-f3dc-4690-94f6-26d2913225e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:08.109063 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cb312-f3dc-4690-94f6-26d2913225e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:08.109341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cb312-f3dc-4690-94f6-26d2913225e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:08.109602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47cb312-f3dc-4690-94f6-26d2913225e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:08.110258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47cb312-f3dc-4690-94f6-26d2913225e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:08.112147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47cb312-f3dc-4690-94f6-26d2913225e4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:08.132882 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1173/2339] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:08 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:08.136862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d98064-bd72-40d3-aff8-2e22131b92de admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:08.147812 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d98064-bd72-40d3-aff8-2e22131b92de admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:08.152352 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75671337-de6c-48d8-9449-2ab312d93ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:08.152931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75671337-de6c-48d8-9449-2ab312d93ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:08.153205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75671337-de6c-48d8-9449-2ab312d93ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:08.153471 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75671337-de6c-48d8-9449-2ab312d93ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:08.154101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75671337-de6c-48d8-9449-2ab312d93ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:08.156896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75671337-de6c-48d8-9449-2ab312d93ccc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:08.167710 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1167/2340] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:53:08.171905 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47cb312-f3dc-4690-94f6-26d2913225e4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:08.183451 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47cb312-f3dc-4690-94f6-26d2913225e4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:08.188386 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aacf1-61ad-4f39-8114-9bf5abfcb82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:08.189094 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aacf1-61ad-4f39-8114-9bf5abfcb82f admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:08.189469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aacf1-61ad-4f39-8114-9bf5abfcb82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:08.189767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aacf1-61ad-4f39-8114-9bf5abfcb82f admin admin] PATH_INFO: `/v3/projects/5fc6f2f2783c4dc3b7ffbfa842e06771/users/32c54a25b2f84592b3689b4ac648fc2a/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:08.207740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277aacf1-61ad-4f39-8114-9bf5abfcb82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc6f2f2783c4dc3b7ffbfa842e06771, user_id=32c54a25b2f84592b3689b4ac648fc2a, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:08.210698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277aacf1-61ad-4f39-8114-9bf5abfcb82f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:08.225883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1174/2341] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:08 2022] PUT /identity/v3/projects/5fc6f2f2783c4dc3b7ffbfa842e06771/users/32c54a25b2f84592b3689b4ac648fc2a/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:08.229853 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75671337-de6c-48d8-9449-2ab312d93ccc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:08.239858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75671337-de6c-48d8-9449-2ab312d93ccc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:08.278498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:08.305056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51e25d-cd41-4573-a093-bfdba7896319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:08.305721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51e25d-cd41-4573-a093-bfdba7896319 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:08.306005 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51e25d-cd41-4573-a093-bfdba7896319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:08.306341 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51e25d-cd41-4573-a093-bfdba7896319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:08.307019 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51e25d-cd41-4573-a093-bfdba7896319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:08.308705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51e25d-cd41-4573-a093-bfdba7896319 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:08.323783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1168/2342] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:08.324953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aacf1-61ad-4f39-8114-9bf5abfcb82f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:08.336231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aacf1-61ad-4f39-8114-9bf5abfcb82f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:08.340947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010213c-5992-458d-8a03-7e125d379ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:08.341593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010213c-5992-458d-8a03-7e125d379ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:08.341854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010213c-5992-458d-8a03-7e125d379ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:08.348322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010213c-5992-458d-8a03-7e125d379ae9 admin admin] PATH_INFO: `/v3/projects/5fc6f2f2783c4dc3b7ffbfa842e06771/users/32c54a25b2f84592b3689b4ac648fc2a/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:08.348322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010213c-5992-458d-8a03-7e125d379ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc6f2f2783c4dc3b7ffbfa842e06771, user_id=32c54a25b2f84592b3689b4ac648fc2a, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:08.348322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010213c-5992-458d-8a03-7e125d379ae9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:08.358744 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1175/2343] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:08 2022] PUT /identity/v3/projects/5fc6f2f2783c4dc3b7ffbfa842e06771/users/32c54a25b2f84592b3689b4ac648fc2a/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:53:08.369962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f4e52-32e4-4aa6-8bc1-610f0abb27a6 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:08.370289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f4e52-32e4-4aa6-8bc1-610f0abb27a6 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:08.370568 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f4e52-32e4-4aa6-8bc1-610f0abb27a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:08.399118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-c80f4e52-32e4-4aa6-8bc1-610f0abb27a6 None None] MFA Rules not processed for user `32c54a25b2f84592b3689b4ac648fc2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:08.400324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c80f4e52-32e4-4aa6-8bc1-610f0abb27a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:08.467421 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1169/2344] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:08.478715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010213c-5992-458d-8a03-7e125d379ae9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:08.480455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c010213c-5992-458d-8a03-7e125d379ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:08.538816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010213c-5992-458d-8a03-7e125d379ae9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:08.542447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca608b4e-8495-40ce-897f-038b8ad7e0cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:08.543043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca608b4e-8495-40ce-897f-038b8ad7e0cd service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:08.543319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca608b4e-8495-40ce-897f-038b8ad7e0cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:08.543588 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca608b4e-8495-40ce-897f-038b8ad7e0cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:08.552767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca608b4e-8495-40ce-897f-038b8ad7e0cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:08.554191 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca608b4e-8495-40ce-897f-038b8ad7e0cd service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:08.564243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1176/2345] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:08.605461 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f4e52-32e4-4aa6-8bc1-610f0abb27a6 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:08.606862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c80f4e52-32e4-4aa6-8bc1-610f0abb27a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:08.618867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f4e52-32e4-4aa6-8bc1-610f0abb27a6 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:08.622489 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3b551-9ad5-4a3e-b378-f00bc6ea2877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:08.623081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3b551-9ad5-4a3e-b378-f00bc6ea2877 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:08.623367 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3b551-9ad5-4a3e-b378-f00bc6ea2877 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:08.623638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3b551-9ad5-4a3e-b378-f00bc6ea2877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:08.679242 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3b551-9ad5-4a3e-b378-f00bc6ea2877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:08.680663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3b551-9ad5-4a3e-b378-f00bc6ea2877 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:08.689825 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1170/2346] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:11.580878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca608b4e-8495-40ce-897f-038b8ad7e0cd service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:11.642207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca608b4e-8495-40ce-897f-038b8ad7e0cd service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:11.646809 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:11.647432 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:11.647708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:11.647977 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] PATH_INFO: `/v3/users/b233a73f0f514429a637ff877cf87a7c` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:11.649077 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b233a73f0f514429a637ff877cf87a7c)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:11.651021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:11.652486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:11.699578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b233a73f0f514429a637ff877cf87a7c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:11.700228 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b233a73f0f514429a637ff877cf87a7c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:11.712774 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b233a73f0f514429a637ff877cf87a7c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:11.713427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b233a73f0f514429a637ff877cf87a7c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:11.721247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b233a73f0f514429a637ff877cf87a7c'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:11.735211 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1177/2347] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:11 2022] DELETE /identity/v3/users/b233a73f0f514429a637ff877cf87a7c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:11.739813 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6781f-c6e5-4a2a-85c8-d6abc283885f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:11.740280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6781f-c6e5-4a2a-85c8-d6abc283885f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:11.740503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6781f-c6e5-4a2a-85c8-d6abc283885f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:11.760434 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-4bb6781f-c6e5-4a2a-85c8-d6abc283885f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:11.761734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4bb6781f-c6e5-4a2a-85c8-d6abc283885f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:11.815730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:11.824770 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1171/2348] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:11.834923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:11.836618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:11.900406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd9646-e636-4dd4-be67-26e0d7aacaa7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:11.904017 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f7097-6023-4918-86bc-ebf230040f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:11.904617 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f7097-6023-4918-86bc-ebf230040f59 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:11.904898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f7097-6023-4918-86bc-ebf230040f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:11.905169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f7097-6023-4918-86bc-ebf230040f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:11.914607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f7097-6023-4918-86bc-ebf230040f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:11.916169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f7097-6023-4918-86bc-ebf230040f59 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:11.926275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1178/2349] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:53:12.192287 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6781f-c6e5-4a2a-85c8-d6abc283885f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:12.193797 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-4bb6781f-c6e5-4a2a-85c8-d6abc283885f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:12.206349 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6781f-c6e5-4a2a-85c8-d6abc283885f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:12.211269 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:12.212335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:12.212909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:12.213186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] PATH_INFO: `/v3/projects/29b0591db51e4ee5b2321c806614a577` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:12.214410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] RBAC: Authorizing `identity:delete_project(project_id=29b0591db51e4ee5b2321c806614a577)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:12.216346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:12.264155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29b0591db51e4ee5b2321c806614a577', 'request_id': 'req-e6559bda-b7be-4953-8444-5094f45de77c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.271995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29b0591db51e4ee5b2321c806614a577', 'request_id': 'req-e6559bda-b7be-4953-8444-5094f45de77c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.278365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29b0591db51e4ee5b2321c806614a577', 'request_id': 'req-e6559bda-b7be-4953-8444-5094f45de77c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.289529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] The token cache is being invalidate because project 29b0591db51e4ee5b2321c806614a577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:12.289839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.290468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.291928 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1172/2350] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:12 2022] DELETE /identity/v3/projects/29b0591db51e4ee5b2321c806614a577 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:12.295948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f7097-6023-4918-86bc-ebf230040f59 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:12.306122 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f7097-6023-4918-86bc-ebf230040f59 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:12.363454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:12.364076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:12.364358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:12.364626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] PATH_INFO: `/v3/users/7d826072fdfb4222bf2127c1f8af785e` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:12.365740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d826072fdfb4222bf2127c1f8af785e)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:12.367668 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:12.389928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:12.414564 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d826072fdfb4222bf2127c1f8af785e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.415229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d826072fdfb4222bf2127c1f8af785e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.427680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d826072fdfb4222bf2127c1f8af785e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.428340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d826072fdfb4222bf2127c1f8af785e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.435172 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d826072fdfb4222bf2127c1f8af785e'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.447402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1179/2351] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:12 2022] DELETE /identity/v3/users/7d826072fdfb4222bf2127c1f8af785e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:12.716212 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:12.717647 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:12.731410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6559bda-b7be-4953-8444-5094f45de77c admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:12.791249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:12.791968 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:12.792140 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:12.792404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] PATH_INFO: `/v3/projects/0d077c96b2604548a50bf9d12b17cd88` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:12.793502 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d077c96b2604548a50bf9d12b17cd88)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:12.795406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:12.841404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d077c96b2604548a50bf9d12b17cd88', 'request_id': 'req-d30a84c9-5066-4740-b407-45f2b0e2c082'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.848830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d077c96b2604548a50bf9d12b17cd88', 'request_id': 'req-d30a84c9-5066-4740-b407-45f2b0e2c082'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.854902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d077c96b2604548a50bf9d12b17cd88', 'request_id': 'req-d30a84c9-5066-4740-b407-45f2b0e2c082'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.865799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] The token cache is being invalidate because project 0d077c96b2604548a50bf9d12b17cd88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:12.866085 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.866768 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:12.868233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1173/2352] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:12 2022] DELETE /identity/v3/projects/0d077c96b2604548a50bf9d12b17cd88 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:12.872402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:12.873882 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:12.887009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b34b8-b41a-49a2-832e-5bc92aa16e65 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:12.947468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d729d-f55c-41dd-905a-c8eb78bce33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:12.948089 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d729d-f55c-41dd-905a-c8eb78bce33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:12.948370 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d729d-f55c-41dd-905a-c8eb78bce33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:12.948643 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d729d-f55c-41dd-905a-c8eb78bce33a admin admin] PATH_INFO: `/v3/users/32c54a25b2f84592b3689b4ac648fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:12.949760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d729d-f55c-41dd-905a-c8eb78bce33a admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c54a25b2f84592b3689b4ac648fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:12.951702 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d729d-f55c-41dd-905a-c8eb78bce33a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:13.002497 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-de4d729d-f55c-41dd-905a-c8eb78bce33a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c54a25b2f84592b3689b4ac648fc2a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:13.003157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-de4d729d-f55c-41dd-905a-c8eb78bce33a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c54a25b2f84592b3689b4ac648fc2a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:13.015860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-de4d729d-f55c-41dd-905a-c8eb78bce33a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c54a25b2f84592b3689b4ac648fc2a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:13.016771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-de4d729d-f55c-41dd-905a-c8eb78bce33a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c54a25b2f84592b3689b4ac648fc2a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:13.024217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-de4d729d-f55c-41dd-905a-c8eb78bce33a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c54a25b2f84592b3689b4ac648fc2a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:13.047201 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1180/2353] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:12 2022] DELETE /identity/v3/users/32c54a25b2f84592b3689b4ac648fc2a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:13.347114 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:13.348602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:13.361714 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a84c9-5066-4740-b407-45f2b0e2c082 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:13.371118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:13.421181 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:13.421881 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:13.422136 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:13.422435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] PATH_INFO: `/v3/projects/5fc6f2f2783c4dc3b7ffbfa842e06771` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:13.423540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fc6f2f2783c4dc3b7ffbfa842e06771)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:13.425433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:13.470809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fc6f2f2783c4dc3b7ffbfa842e06771', 'request_id': 'req-b06908c6-cf90-46c6-9d68-c4538887f975'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:13.477761 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fc6f2f2783c4dc3b7ffbfa842e06771', 'request_id': 'req-b06908c6-cf90-46c6-9d68-c4538887f975'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:13.483786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fc6f2f2783c4dc3b7ffbfa842e06771', 'request_id': 'req-b06908c6-cf90-46c6-9d68-c4538887f975'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:13.494134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] The token cache is being invalidate because project 5fc6f2f2783c4dc3b7ffbfa842e06771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:13.494493 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:13.495351 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-b06908c6-cf90-46c6-9d68-c4538887f975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:13.497441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1174/2354] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:13 2022] DELETE /identity/v3/projects/5fc6f2f2783c4dc3b7ffbfa842e06771 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:13.503312 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498266a3-03be-455e-a702-4c6cdd97fc9f None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:13.503601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498266a3-03be-455e-a702-4c6cdd97fc9f None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:13.503875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498266a3-03be-455e-a702-4c6cdd97fc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:13.524397 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-498266a3-03be-455e-a702-4c6cdd97fc9f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:13.525529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-498266a3-03be-455e-a702-4c6cdd97fc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:13.539293 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:13.585952 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1181/2355] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:13.594950 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a3ced-ae79-4ea3-8997-cd76629b7179 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:13.595250 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a3ced-ae79-4ea3-8997-cd76629b7179 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:13.595513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a3ced-ae79-4ea3-8997-cd76629b7179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:13.614957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8b5a3ced-ae79-4ea3-8997-cd76629b7179 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:13.616132 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8b5a3ced-ae79-4ea3-8997-cd76629b7179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:13.624786 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1175/2356] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:13.628923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498266a3-03be-455e-a702-4c6cdd97fc9f None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:13.638634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498266a3-03be-455e-a702-4c6cdd97fc9f None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:13.643966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c45d89-1b9e-4f93-8256-11c7df529a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:13.644558 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c45d89-1b9e-4f93-8256-11c7df529a04 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:13.644838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c45d89-1b9e-4f93-8256-11c7df529a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:13.645138 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c45d89-1b9e-4f93-8256-11c7df529a04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:13.645782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c45d89-1b9e-4f93-8256-11c7df529a04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:53:13.646081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c45d89-1b9e-4f93-8256-11c7df529a04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:13.647701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c45d89-1b9e-4f93-8256-11c7df529a04 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:13.659631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1182/2357] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:53:13.664056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a3ced-ae79-4ea3-8997-cd76629b7179 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:13.673178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a3ced-ae79-4ea3-8997-cd76629b7179 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:13.677833 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c72fa-3450-4983-a1ac-d1a87d2ba34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:13.678468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c72fa-3450-4983-a1ac-d1a87d2ba34d admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:13.678752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c72fa-3450-4983-a1ac-d1a87d2ba34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:13.679025 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c72fa-3450-4983-a1ac-d1a87d2ba34d admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:13.679661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c72fa-3450-4983-a1ac-d1a87d2ba34d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:13.681547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c72fa-3450-4983-a1ac-d1a87d2ba34d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:13.709785 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1176/2358] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:13 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:13.713849 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c45d89-1b9e-4f93-8256-11c7df529a04 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:13.724216 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c45d89-1b9e-4f93-8256-11c7df529a04 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:13.779943 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a0221-b1cd-4609-8e99-ec264855b8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:13.780820 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a0221-b1cd-4609-8e99-ec264855b8af admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:13.781102 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a0221-b1cd-4609-8e99-ec264855b8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:13.781368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a0221-b1cd-4609-8e99-ec264855b8af admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:13.781986 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a0221-b1cd-4609-8e99-ec264855b8af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:13.783914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a0221-b1cd-4609-8e99-ec264855b8af admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:13.804296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1183/2359] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:13 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:13.808452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c72fa-3450-4983-a1ac-d1a87d2ba34d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:13.819869 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c72fa-3450-4983-a1ac-d1a87d2ba34d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:13.824473 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576fd16-9f44-418a-9317-513251b3451f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:13.825084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576fd16-9f44-418a-9317-513251b3451f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:13.825359 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576fd16-9f44-418a-9317-513251b3451f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:13.825623 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576fd16-9f44-418a-9317-513251b3451f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:13.826299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576fd16-9f44-418a-9317-513251b3451f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:13.828496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576fd16-9f44-418a-9317-513251b3451f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:13.838313 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1177/2360] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:13.842308 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a0221-b1cd-4609-8e99-ec264855b8af admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:13.852301 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a0221-b1cd-4609-8e99-ec264855b8af admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:13.856847 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849b0f9-6e58-4893-81e9-a455219c0a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:13.857431 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849b0f9-6e58-4893-81e9-a455219c0a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:13.857712 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849b0f9-6e58-4893-81e9-a455219c0a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:13.857984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849b0f9-6e58-4893-81e9-a455219c0a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:13.858741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849b0f9-6e58-4893-81e9-a455219c0a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:13.860461 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849b0f9-6e58-4893-81e9-a455219c0a90 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:13.871425 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1184/2361] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:13.876345 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576fd16-9f44-418a-9317-513251b3451f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:13.886312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576fd16-9f44-418a-9317-513251b3451f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:13.891935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaa97b-f2eb-402d-a2ca-79eaafe5203b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:13.892573 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaa97b-f2eb-402d-a2ca-79eaafe5203b admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:13.892847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaa97b-f2eb-402d-a2ca-79eaafe5203b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:13.893116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaa97b-f2eb-402d-a2ca-79eaafe5203b admin admin] PATH_INFO: `/v3/projects/a98f507b288c4fcbb056b8de2663d63b/users/140cacb937df433c804b4362e20128a8/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:13.908055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaa97b-f2eb-402d-a2ca-79eaafe5203b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98f507b288c4fcbb056b8de2663d63b, user_id=140cacb937df433c804b4362e20128a8, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:13.910830 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaa97b-f2eb-402d-a2ca-79eaafe5203b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:13.923953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1178/2362] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:13 2022] PUT /identity/v3/projects/a98f507b288c4fcbb056b8de2663d63b/users/140cacb937df433c804b4362e20128a8/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:13.936423 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921e19f-1667-4244-8f45-244f0ff22368 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:13.937470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921e19f-1667-4244-8f45-244f0ff22368 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:13.937470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921e19f-1667-4244-8f45-244f0ff22368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:13.965539 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-9921e19f-1667-4244-8f45-244f0ff22368 None None] MFA Rules not processed for user `140cacb937df433c804b4362e20128a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:13.966771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9921e19f-1667-4244-8f45-244f0ff22368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:14.031386 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1185/2363] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:14.042692 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaa97b-f2eb-402d-a2ca-79eaafe5203b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:14.044346 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-2feaa97b-f2eb-402d-a2ca-79eaafe5203b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:14.089226 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:14.104147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaa97b-f2eb-402d-a2ca-79eaafe5203b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:14.107777 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526e641-7759-4dfe-bbc1-d6a06c97b861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:14.108372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526e641-7759-4dfe-bbc1-d6a06c97b861 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:14.109213 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526e641-7759-4dfe-bbc1-d6a06c97b861 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:14.109553 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526e641-7759-4dfe-bbc1-d6a06c97b861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:14.117678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e526e641-7759-4dfe-bbc1-d6a06c97b861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:14.119305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e526e641-7759-4dfe-bbc1-d6a06c97b861 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:14.129220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1179/2364] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:14.149740 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921e19f-1667-4244-8f45-244f0ff22368 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:14.167663 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:14.223788 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921e19f-1667-4244-8f45-244f0ff22368 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:14.228575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:14.229220 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:14.229892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:14.229892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] PATH_INFO: `/v3/users/140cacb937df433c804b4362e20128a8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:14.230919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=140cacb937df433c804b4362e20128a8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:14.232842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:14.288472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '140cacb937df433c804b4362e20128a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:14.289151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '140cacb937df433c804b4362e20128a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:14.301928 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '140cacb937df433c804b4362e20128a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:14.306615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '140cacb937df433c804b4362e20128a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:14.315151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '140cacb937df433c804b4362e20128a8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:14.330003 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1186/2365] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:14 2022] DELETE /identity/v3/users/140cacb937df433c804b4362e20128a8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:14.334552 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b39ff-2b65-4875-a735-4bc6fd362bff None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:14.334863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b39ff-2b65-4875-a735-4bc6fd362bff None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:14.335311 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b39ff-2b65-4875-a735-4bc6fd362bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:14.354587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-281b39ff-2b65-4875-a735-4bc6fd362bff None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:14.355799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-281b39ff-2b65-4875-a735-4bc6fd362bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:14.418565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1180/2366] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:53:14.429185 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:14.430670 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:14.496425 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63256140-aa37-452d-971a-37def66ee763 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:14.500495 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963c1d9-3c89-48df-8d92-a5acb68f7724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:14.501143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963c1d9-3c89-48df-8d92-a5acb68f7724 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:14.501422 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963c1d9-3c89-48df-8d92-a5acb68f7724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:14.501688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963c1d9-3c89-48df-8d92-a5acb68f7724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:14.510795 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963c1d9-3c89-48df-8d92-a5acb68f7724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:14.512195 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963c1d9-3c89-48df-8d92-a5acb68f7724 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:14.522802 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1187/2367] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:14.792379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b39ff-2b65-4875-a735-4bc6fd362bff None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:14.793820 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-281b39ff-2b65-4875-a735-4bc6fd362bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:14.805734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b39ff-2b65-4875-a735-4bc6fd362bff None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:14.810300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:14.810929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:14.811207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:14.811475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] PATH_INFO: `/v3/projects/a98f507b288c4fcbb056b8de2663d63b` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:14.812915 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98f507b288c4fcbb056b8de2663d63b)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:14.814876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:14.862521 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98f507b288c4fcbb056b8de2663d63b', 'request_id': 'req-57a9ad12-7d69-4535-b050-c880a3dcbc78'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:14.869995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98f507b288c4fcbb056b8de2663d63b', 'request_id': 'req-57a9ad12-7d69-4535-b050-c880a3dcbc78'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:14.876546 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98f507b288c4fcbb056b8de2663d63b', 'request_id': 'req-57a9ad12-7d69-4535-b050-c880a3dcbc78'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:14.886989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] The token cache is being invalidate because project a98f507b288c4fcbb056b8de2663d63b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:14.887273 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:14.887890 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-57a9ad12-7d69-4535-b050-c880a3dcbc78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:14.889378 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1181/2368] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:14 2022] DELETE /identity/v3/projects/a98f507b288c4fcbb056b8de2663d63b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:14.894867 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf472-f1ee-4048-bbb2-93b54d57e10c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:14.895149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf472-f1ee-4048-bbb2-93b54d57e10c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:14.895418 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf472-f1ee-4048-bbb2-93b54d57e10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:14.915140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-38ecf472-f1ee-4048-bbb2-93b54d57e10c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:14.916263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-38ecf472-f1ee-4048-bbb2-93b54d57e10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:14.947614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:14.976678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1188/2369] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:14.984778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f55ca9-85cc-4187-913f-ee0dba754f38 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:14.985058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f55ca9-85cc-4187-913f-ee0dba754f38 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:14.985330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f55ca9-85cc-4187-913f-ee0dba754f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.005287 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-83f55ca9-85cc-4187-913f-ee0dba754f38 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:15.006504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-83f55ca9-85cc-4187-913f-ee0dba754f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:15.014299 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1182/2370] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:15.018550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf472-f1ee-4048-bbb2-93b54d57e10c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:15.027962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf472-f1ee-4048-bbb2-93b54d57e10c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:15.033236 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f81087-79a0-44dc-8ce0-f0baaa65c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:15.033823 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f81087-79a0-44dc-8ce0-f0baaa65c963 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.034099 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f81087-79a0-44dc-8ce0-f0baaa65c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.034471 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f81087-79a0-44dc-8ce0-f0baaa65c963 admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.035196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f81087-79a0-44dc-8ce0-f0baaa65c963 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:53:15.035665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f81087-79a0-44dc-8ce0-f0baaa65c963 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:15.037547 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f81087-79a0-44dc-8ce0-f0baaa65c963 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:15.051222 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1189/2371] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:53:15.055553 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f55ca9-85cc-4187-913f-ee0dba754f38 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:15.067315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f55ca9-85cc-4187-913f-ee0dba754f38 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:15.072017 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae678549-e4bd-4593-94da-51cb889b2de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:15.072618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae678549-e4bd-4593-94da-51cb889b2de4 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.072896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae678549-e4bd-4593-94da-51cb889b2de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.073165 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae678549-e4bd-4593-94da-51cb889b2de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.073810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae678549-e4bd-4593-94da-51cb889b2de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:15.075731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae678549-e4bd-4593-94da-51cb889b2de4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:15.105579 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1183/2372] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:15 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:15.110455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f81087-79a0-44dc-8ce0-f0baaa65c963 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:15.120290 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f81087-79a0-44dc-8ce0-f0baaa65c963 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:15.179239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb001350-f9b6-4331-a9db-e4b2d52f7245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:15.179846 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb001350-f9b6-4331-a9db-e4b2d52f7245 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.180123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb001350-f9b6-4331-a9db-e4b2d52f7245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.180390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb001350-f9b6-4331-a9db-e4b2d52f7245 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.181018 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb001350-f9b6-4331-a9db-e4b2d52f7245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:15.182938 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb001350-f9b6-4331-a9db-e4b2d52f7245 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:15.204924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1190/2373] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:15 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:53:15.209508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae678549-e4bd-4593-94da-51cb889b2de4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:15.224373 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae678549-e4bd-4593-94da-51cb889b2de4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:15.229347 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9849f-f6fc-46e1-ada8-016ccb0a0924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:15.230157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9849f-f6fc-46e1-ada8-016ccb0a0924 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.230469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9849f-f6fc-46e1-ada8-016ccb0a0924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.230740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9849f-f6fc-46e1-ada8-016ccb0a0924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.231381 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9849f-f6fc-46e1-ada8-016ccb0a0924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:15.232974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9849f-f6fc-46e1-ada8-016ccb0a0924 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:15.244227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1184/2374] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:15.248465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb001350-f9b6-4331-a9db-e4b2d52f7245 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:15.260082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb001350-f9b6-4331-a9db-e4b2d52f7245 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:15.265519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d1b3b-0179-4d9d-9d86-e0d1d1eb624a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:15.266367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d1b3b-0179-4d9d-9d86-e0d1d1eb624a admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.266649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d1b3b-0179-4d9d-9d86-e0d1d1eb624a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.266921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d1b3b-0179-4d9d-9d86-e0d1d1eb624a admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.267576 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d1b3b-0179-4d9d-9d86-e0d1d1eb624a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:15.269172 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d1b3b-0179-4d9d-9d86-e0d1d1eb624a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:15.280241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1191/2375] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:15.284931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9849f-f6fc-46e1-ada8-016ccb0a0924 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:15.298734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9849f-f6fc-46e1-ada8-016ccb0a0924 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:15.304799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3945be-53b0-4136-a9ae-365559329bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:15.305733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3945be-53b0-4136-a9ae-365559329bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.306015 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3945be-53b0-4136-a9ae-365559329bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.306413 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3945be-53b0-4136-a9ae-365559329bf0 admin admin] PATH_INFO: `/v3/projects/11922c4b1f9349dba8a7ed22ee7c3d0e/users/eaee634ad3504c26992a8a0d1fbf7f79/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.322667 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3945be-53b0-4136-a9ae-365559329bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11922c4b1f9349dba8a7ed22ee7c3d0e, user_id=eaee634ad3504c26992a8a0d1fbf7f79, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:15.325976 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3945be-53b0-4136-a9ae-365559329bf0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:15.342286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1185/2376] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:15 2022] PUT /identity/v3/projects/11922c4b1f9349dba8a7ed22ee7c3d0e/users/eaee634ad3504c26992a8a0d1fbf7f79/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:15.353242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca7042-a638-4732-b5fb-09f800f98604 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.353531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca7042-a638-4732-b5fb-09f800f98604 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.353824 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca7042-a638-4732-b5fb-09f800f98604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.386760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-1dca7042-a638-4732-b5fb-09f800f98604 None None] MFA Rules not processed for user `eaee634ad3504c26992a8a0d1fbf7f79`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:15.387960 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-1dca7042-a638-4732-b5fb-09f800f98604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:15.459665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1192/2377] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:15.464249 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3945be-53b0-4136-a9ae-365559329bf0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:15.471560 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:15.543060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3945be-53b0-4136-a9ae-365559329bf0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:15.548450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91086b36-7f89-4880-942c-1fa81e0e9473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:15.549106 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91086b36-7f89-4880-942c-1fa81e0e9473 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.549326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91086b36-7f89-4880-942c-1fa81e0e9473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.549603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91086b36-7f89-4880-942c-1fa81e0e9473 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.550261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91086b36-7f89-4880-942c-1fa81e0e9473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:15.552227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91086b36-7f89-4880-942c-1fa81e0e9473 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:15.583498 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1186/2378] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:15 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:53:15.585627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca7042-a638-4732-b5fb-09f800f98604 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:15.596429 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca7042-a638-4732-b5fb-09f800f98604 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:15.632958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:15.667452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ee785-58f1-42b4-92f5-cd8628e3c496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:15.668518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ee785-58f1-42b4-92f5-cd8628e3c496 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.668806 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ee785-58f1-42b4-92f5-cd8628e3c496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.669083 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ee785-58f1-42b4-92f5-cd8628e3c496 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.670105 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ee785-58f1-42b4-92f5-cd8628e3c496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:15.672778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ee785-58f1-42b4-92f5-cd8628e3c496 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:15.700428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1193/2379] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:15 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:15.706155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91086b36-7f89-4880-942c-1fa81e0e9473 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:15.719137 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91086b36-7f89-4880-942c-1fa81e0e9473 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:15.724323 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e08f1-8193-4862-ad2f-f6bbc4037399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:15.724941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e08f1-8193-4862-ad2f-f6bbc4037399 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.725220 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e08f1-8193-4862-ad2f-f6bbc4037399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.725581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e08f1-8193-4862-ad2f-f6bbc4037399 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.726266 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e08f1-8193-4862-ad2f-f6bbc4037399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:15.727910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e08f1-8193-4862-ad2f-f6bbc4037399 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:15.744096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1187/2380] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:15.748677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ee785-58f1-42b4-92f5-cd8628e3c496 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:15.760311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ee785-58f1-42b4-92f5-cd8628e3c496 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:15.765524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79380c30-fadb-48a8-a1ad-07485b5a005e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:15.766349 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79380c30-fadb-48a8-a1ad-07485b5a005e admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.766698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79380c30-fadb-48a8-a1ad-07485b5a005e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.767036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79380c30-fadb-48a8-a1ad-07485b5a005e admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.767754 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79380c30-fadb-48a8-a1ad-07485b5a005e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:15.769424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79380c30-fadb-48a8-a1ad-07485b5a005e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:15.790515 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1194/2381] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:15.792004 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e08f1-8193-4862-ad2f-f6bbc4037399 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:15.806355 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e08f1-8193-4862-ad2f-f6bbc4037399 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:15.813342 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157ba48-8ba3-4390-ba1c-c05e7ec06c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:15.814070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157ba48-8ba3-4390-ba1c-c05e7ec06c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.814649 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157ba48-8ba3-4390-ba1c-c05e7ec06c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.815195 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157ba48-8ba3-4390-ba1c-c05e7ec06c02 admin admin] PATH_INFO: `/v3/projects/e3d0a0646f604bec80871972d81c53dd/users/f891964c03394f739b174303f52c1851/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.834002 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157ba48-8ba3-4390-ba1c-c05e7ec06c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d0a0646f604bec80871972d81c53dd, user_id=f891964c03394f739b174303f52c1851, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:15.837001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157ba48-8ba3-4390-ba1c-c05e7ec06c02 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:15.852360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1188/2382] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:15 2022] PUT /identity/v3/projects/e3d0a0646f604bec80871972d81c53dd/users/f891964c03394f739b174303f52c1851/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:15.937916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b5548-a651-41d4-a8d7-6fd015666c7b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:15.938531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b5548-a651-41d4-a8d7-6fd015666c7b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:15.938860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b5548-a651-41d4-a8d7-6fd015666c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:15.967275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f62b5548-a651-41d4-a8d7-6fd015666c7b None None] MFA Rules not processed for user `f891964c03394f739b174303f52c1851`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:15.969223 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f62b5548-a651-41d4-a8d7-6fd015666c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:16.036285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1195/2383] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:16.040751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157ba48-8ba3-4390-ba1c-c05e7ec06c02 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:16.110123 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157ba48-8ba3-4390-ba1c-c05e7ec06c02 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:16.115305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8fea4-5f69-4ac4-860f-904b703cdb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:16.115910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8fea4-5f69-4ac4-860f-904b703cdb60 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:16.116193 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8fea4-5f69-4ac4-860f-904b703cdb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:16.116503 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8fea4-5f69-4ac4-860f-904b703cdb60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:16.117184 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8fea4-5f69-4ac4-860f-904b703cdb60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:16.119122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8fea4-5f69-4ac4-860f-904b703cdb60 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:16.150899 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1189/2384] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:16 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:16.152441 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b5548-a651-41d4-a8d7-6fd015666c7b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:16.164933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b5548-a651-41d4-a8d7-6fd015666c7b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:16.224953 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36049170-d092-49ea-b5e2-73d6a1de2840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:16.225593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36049170-d092-49ea-b5e2-73d6a1de2840 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:16.225852 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36049170-d092-49ea-b5e2-73d6a1de2840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:16.226120 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36049170-d092-49ea-b5e2-73d6a1de2840 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:16.226783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36049170-d092-49ea-b5e2-73d6a1de2840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:16.229095 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36049170-d092-49ea-b5e2-73d6a1de2840 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:16.255435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1196/2385] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:16 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:16.256642 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8fea4-5f69-4ac4-860f-904b703cdb60 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:16.267973 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8fea4-5f69-4ac4-860f-904b703cdb60 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:16.275010 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866e335-0ea1-4c8b-b030-d6a7140b7b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:16.275948 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866e335-0ea1-4c8b-b030-d6a7140b7b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:16.276232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866e335-0ea1-4c8b-b030-d6a7140b7b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:16.276507 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866e335-0ea1-4c8b-b030-d6a7140b7b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:16.277173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866e335-0ea1-4c8b-b030-d6a7140b7b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:16.278951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866e335-0ea1-4c8b-b030-d6a7140b7b4e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:16.292478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1190/2386] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:16.296398 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36049170-d092-49ea-b5e2-73d6a1de2840 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:16.312234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36049170-d092-49ea-b5e2-73d6a1de2840 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:16.318206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a87a2-42c6-46e6-85d7-4fd1905e72fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:16.318971 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a87a2-42c6-46e6-85d7-4fd1905e72fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:16.319447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a87a2-42c6-46e6-85d7-4fd1905e72fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:16.319794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a87a2-42c6-46e6-85d7-4fd1905e72fa admin admin] PATH_INFO: `/v3/projects/f83db658b5494a01bb535d15bc3e8299/users/f3e02552782a48da8e94e8a8ee0b5eb6/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:16.340958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a87a2-42c6-46e6-85d7-4fd1905e72fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f83db658b5494a01bb535d15bc3e8299, user_id=f3e02552782a48da8e94e8a8ee0b5eb6, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:16.345274 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a87a2-42c6-46e6-85d7-4fd1905e72fa admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:16.364829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1197/2387] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:16 2022] PUT /identity/v3/projects/f83db658b5494a01bb535d15bc3e8299/users/f3e02552782a48da8e94e8a8ee0b5eb6/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:16.369115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866e335-0ea1-4c8b-b030-d6a7140b7b4e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:16.385590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866e335-0ea1-4c8b-b030-d6a7140b7b4e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:16.411792 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:16.447982 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24cffb-c3be-4446-8931-4250236becf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:16.448595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24cffb-c3be-4446-8931-4250236becf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:16.448875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24cffb-c3be-4446-8931-4250236becf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:16.449146 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24cffb-c3be-4446-8931-4250236becf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:16.449823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24cffb-c3be-4446-8931-4250236becf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:16.451468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24cffb-c3be-4446-8931-4250236becf2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:16.463660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1191/2388] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:16.468836 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a87a2-42c6-46e6-85d7-4fd1905e72fa admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:16.479609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a87a2-42c6-46e6-85d7-4fd1905e72fa admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:16.484827 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151f43e-199c-4b54-a1e9-3ec1d13dddab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:16.485494 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151f43e-199c-4b54-a1e9-3ec1d13dddab admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:16.485776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151f43e-199c-4b54-a1e9-3ec1d13dddab admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:16.486047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151f43e-199c-4b54-a1e9-3ec1d13dddab admin admin] PATH_INFO: `/v3/projects/f83db658b5494a01bb535d15bc3e8299/users/f3e02552782a48da8e94e8a8ee0b5eb6/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:16.488067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151f43e-199c-4b54-a1e9-3ec1d13dddab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f83db658b5494a01bb535d15bc3e8299, user_id=f3e02552782a48da8e94e8a8ee0b5eb6, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:16.490648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151f43e-199c-4b54-a1e9-3ec1d13dddab admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:16.493031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:16.504450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1198/2389] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:16 2022] PUT /identity/v3/projects/f83db658b5494a01bb535d15bc3e8299/users/f3e02552782a48da8e94e8a8ee0b5eb6/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:53:16.514632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba70f78-b53b-4b75-a03e-d6a5eebe4cfb None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:16.515442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba70f78-b53b-4b75-a03e-d6a5eebe4cfb None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:16.515875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba70f78-b53b-4b75-a03e-d6a5eebe4cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:16.546678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-6ba70f78-b53b-4b75-a03e-d6a5eebe4cfb None None] MFA Rules not processed for user `f3e02552782a48da8e94e8a8ee0b5eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:16.548061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6ba70f78-b53b-4b75-a03e-d6a5eebe4cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:16.615752 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1192/2390] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:16.627646 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151f43e-199c-4b54-a1e9-3ec1d13dddab admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:16.629310 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-4151f43e-199c-4b54-a1e9-3ec1d13dddab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:16.685642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151f43e-199c-4b54-a1e9-3ec1d13dddab admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:16.689779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84356c-0608-4047-9c0d-695d9cfa803b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:16.690394 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84356c-0608-4047-9c0d-695d9cfa803b service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:16.690669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84356c-0608-4047-9c0d-695d9cfa803b service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:16.690934 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84356c-0608-4047-9c0d-695d9cfa803b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:16.744225 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84356c-0608-4047-9c0d-695d9cfa803b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:16.745631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84356c-0608-4047-9c0d-695d9cfa803b service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:16.754775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1199/2391] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:18.151236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3acbc-52bc-4298-994c-4945526e2493 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:18.151697 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3acbc-52bc-4298-994c-4945526e2493 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:18.151802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3acbc-52bc-4298-994c-4945526e2493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:18.171766 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f7c3acbc-52bc-4298-994c-4945526e2493 None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:18.173428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f7c3acbc-52bc-4298-994c-4945526e2493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:18.242304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1193/2392] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:53:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:18.266514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84356c-0608-4047-9c0d-695d9cfa803b service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:18.268753 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5e84356c-0608-4047-9c0d-695d9cfa803b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:18.292228 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3acbc-52bc-4298-994c-4945526e2493 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:18.293618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f7c3acbc-52bc-4298-994c-4945526e2493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:18.305021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3acbc-52bc-4298-994c-4945526e2493 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:18.308636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65318c-b3ca-4973-a6f9-caebd5db989e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:18.309235 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65318c-b3ca-4973-a6f9-caebd5db989e service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:18.309514 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65318c-b3ca-4973-a6f9-caebd5db989e service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:18.309785 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65318c-b3ca-4973-a6f9-caebd5db989e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:18.345566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84356c-0608-4047-9c0d-695d9cfa803b service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:18.350654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3fb5c-5dde-4f10-b7be-914e9e26290a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:18.351256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3fb5c-5dde-4f10-b7be-914e9e26290a service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:18.351533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3fb5c-5dde-4f10-b7be-914e9e26290a service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:18.351798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3fb5c-5dde-4f10-b7be-914e9e26290a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:18.360233 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3fb5c-5dde-4f10-b7be-914e9e26290a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:18.361768 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:18.361898 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3fb5c-5dde-4f10-b7be-914e9e26290a service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:18.370147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1200/2393] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:53:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:53:18.371486 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65318c-b3ca-4973-a6f9-caebd5db989e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:18.372985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65318c-b3ca-4973-a6f9-caebd5db989e service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:18.383283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1194/2394] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:53:18.911580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f42555-b13b-4ef4-9b9b-dabb2741fd83 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:18.911893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f42555-b13b-4ef4-9b9b-dabb2741fd83 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:18.912131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f42555-b13b-4ef4-9b9b-dabb2741fd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:18.932364 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b4f42555-b13b-4ef4-9b9b-dabb2741fd83 None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:18.933373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b4f42555-b13b-4ef4-9b9b-dabb2741fd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:18.934935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65318c-b3ca-4973-a6f9-caebd5db989e service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:18.940575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1201/2395] 10.222.0.20 () {58 vars in 967 bytes} [Mon Oct 24 13:53:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:18.943914 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65318c-b3ca-4973-a6f9-caebd5db989e service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:18.947592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1865130-f67d-46ee-a072-f3f99412d09f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:18.948189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1865130-f67d-46ee-a072-f3f99412d09f service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:18.948467 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1865130-f67d-46ee-a072-f3f99412d09f service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:18.948795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1865130-f67d-46ee-a072-f3f99412d09f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:18.953890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f42555-b13b-4ef4-9b9b-dabb2741fd83 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:18.958404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1865130-f67d-46ee-a072-f3f99412d09f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:18.959811 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1865130-f67d-46ee-a072-f3f99412d09f service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:18.965637 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f42555-b13b-4ef4-9b9b-dabb2741fd83 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:18.969718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1195/2396] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:18.971829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d289fdc-ee1c-4f05-ac94-994e4ab370dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:18.972425 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d289fdc-ee1c-4f05-ac94-994e4ab370dd service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:18.972698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d289fdc-ee1c-4f05-ac94-994e4ab370dd service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:18.972963 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d289fdc-ee1c-4f05-ac94-994e4ab370dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:18.981810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d289fdc-ee1c-4f05-ac94-994e4ab370dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:18.983480 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d289fdc-ee1c-4f05-ac94-994e4ab370dd service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:18.992106 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1202/2397] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:53:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:53:19.238759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df1dfc-43cf-4435-8ee3-6154fecf0135 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:19.239333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df1dfc-43cf-4435-8ee3-6154fecf0135 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:19.239333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df1dfc-43cf-4435-8ee3-6154fecf0135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:19.258858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-a8df1dfc-43cf-4435-8ee3-6154fecf0135 None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:19.260214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a8df1dfc-43cf-4435-8ee3-6154fecf0135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:19.268280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1196/2398] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Oct 24 13:53:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:19.282580 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d289fdc-ee1c-4f05-ac94-994e4ab370dd service swift] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:19.293245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d289fdc-ee1c-4f05-ac94-994e4ab370dd service swift] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:19.298455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708aa14f-99a1-4e63-a3b7-1e948bac0b11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:19.299048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aa14f-99a1-4e63-a3b7-1e948bac0b11 service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:19.299327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aa14f-99a1-4e63-a3b7-1e948bac0b11 service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:19.299601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aa14f-99a1-4e63-a3b7-1e948bac0b11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:19.307584 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708aa14f-99a1-4e63-a3b7-1e948bac0b11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:19.309140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708aa14f-99a1-4e63-a3b7-1e948bac0b11 service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:19.317117 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1203/2399] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:53:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:53:19.634556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb3179-7d70-4762-b773-e310cde6dbb5 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:19.634840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb3179-7d70-4762-b773-e310cde6dbb5 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:19.635112 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb3179-7d70-4762-b773-e310cde6dbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:19.654328 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-41cb3179-7d70-4762-b773-e310cde6dbb5 None None] MFA Rules not processed for user `cb6d3abe410746828fb07621ef1a1a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:19.655824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-41cb3179-7d70-4762-b773-e310cde6dbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:19.662824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1197/2400] 10.222.0.20 () {58 vars in 967 bytes} [Mon Oct 24 13:53:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:19.675287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708aa14f-99a1-4e63-a3b7-1e948bac0b11 service swift] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:19.681733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:19.686046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708aa14f-99a1-4e63-a3b7-1e948bac0b11 service swift] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:19.692998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c9ca0-4580-458c-ae1b-3160cc4d204b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea0d300a51c41fba4d4193dd2251a50', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:19.693584 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c9ca0-4580-458c-ae1b-3160cc4d204b service swift] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:19.693862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c9ca0-4580-458c-ae1b-3160cc4d204b service swift] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:19.694127 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c9ca0-4580-458c-ae1b-3160cc4d204b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:19.702101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c9ca0-4580-458c-ae1b-3160cc4d204b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:19.704467 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c9ca0-4580-458c-ae1b-3160cc4d204b service swift] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:19.712365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1204/2401] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Oct 24 13:53:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:53:21.822538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb3179-7d70-4762-b773-e310cde6dbb5 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:21.823981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-41cb3179-7d70-4762-b773-e310cde6dbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:21.890740 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb3179-7d70-4762-b773-e310cde6dbb5 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:21.895300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:21.896036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:21.896428 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:21.896713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] PATH_INFO: `/v3/users/0d46e4243350431b9ff792b269b9e4be` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:21.897850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d46e4243350431b9ff792b269b9e4be)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:21.899801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:21.948625 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d46e4243350431b9ff792b269b9e4be'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:21.949206 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d46e4243350431b9ff792b269b9e4be'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:21.949797 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d46e4243350431b9ff792b269b9e4be'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:21.957102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d46e4243350431b9ff792b269b9e4be'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:21.971072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d46e4243350431b9ff792b269b9e4be'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:21.985602 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1198/2402] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:21 2022] DELETE /identity/v3/users/0d46e4243350431b9ff792b269b9e4be => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:21.990030 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271875b9-2d2f-4958-9a39-0749adeca252 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:21.990558 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271875b9-2d2f-4958-9a39-0749adeca252 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:21.990694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271875b9-2d2f-4958-9a39-0749adeca252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.009198 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-271875b9-2d2f-4958-9a39-0749adeca252 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:22.010424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-271875b9-2d2f-4958-9a39-0749adeca252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:22.069566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1205/2403] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:22.079190 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:22.080617 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:22.126767 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:22.138449 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c6634-c05c-4473-a782-478c27feda76 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:22.142216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584b8f7-f50a-4608-ab20-4591ee982737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:22.142815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584b8f7-f50a-4608-ab20-4591ee982737 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.143091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584b8f7-f50a-4608-ab20-4591ee982737 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.143360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584b8f7-f50a-4608-ab20-4591ee982737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.152217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584b8f7-f50a-4608-ab20-4591ee982737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:22.153633 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584b8f7-f50a-4608-ab20-4591ee982737 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:22.164295 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1199/2404] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:22.332101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271875b9-2d2f-4958-9a39-0749adeca252 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:22.333504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-271875b9-2d2f-4958-9a39-0749adeca252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:22.346629 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271875b9-2d2f-4958-9a39-0749adeca252 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:22.351326 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:22.352029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.352533 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.352866 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] PATH_INFO: `/v3/projects/9c61c491e2924ffe95acc3f4e189d37d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.354233 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c61c491e2924ffe95acc3f4e189d37d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:22.356396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:22.404258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c61c491e2924ffe95acc3f4e189d37d', 'request_id': 'req-80c90714-01ea-4793-b9a5-869e4c1fc7f4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:22.410690 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c61c491e2924ffe95acc3f4e189d37d', 'request_id': 'req-80c90714-01ea-4793-b9a5-869e4c1fc7f4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:22.418636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c61c491e2924ffe95acc3f4e189d37d', 'request_id': 'req-80c90714-01ea-4793-b9a5-869e4c1fc7f4'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:22.430666 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] The token cache is being invalidate because project 9c61c491e2924ffe95acc3f4e189d37d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:22.430971 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:22.431593 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-80c90714-01ea-4793-b9a5-869e4c1fc7f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:22.433060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1206/2405] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:22 2022] DELETE /identity/v3/projects/9c61c491e2924ffe95acc3f4e189d37d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:22.438340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c582628-677e-4006-817e-b7295f1b7817 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.438627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c582628-677e-4006-817e-b7295f1b7817 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.438901 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c582628-677e-4006-817e-b7295f1b7817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.458814 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-8c582628-677e-4006-817e-b7295f1b7817 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:22.459944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-8c582628-677e-4006-817e-b7295f1b7817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:22.517633 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1200/2406] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:22.526350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec8a56-1bbd-4bbe-9e32-c8102a350942 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.526639 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec8a56-1bbd-4bbe-9e32-c8102a350942 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.526915 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec8a56-1bbd-4bbe-9e32-c8102a350942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.547915 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-08ec8a56-1bbd-4bbe-9e32-c8102a350942 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:22.549114 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-08ec8a56-1bbd-4bbe-9e32-c8102a350942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:22.557001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1207/2407] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:22.561116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c582628-677e-4006-817e-b7295f1b7817 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:22.570855 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c582628-677e-4006-817e-b7295f1b7817 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:22.575345 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46223cd0-790e-4aa3-8a62-5d057c294705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:22.575931 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46223cd0-790e-4aa3-8a62-5d057c294705 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.576208 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46223cd0-790e-4aa3-8a62-5d057c294705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.576471 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46223cd0-790e-4aa3-8a62-5d057c294705 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.577104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46223cd0-790e-4aa3-8a62-5d057c294705 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:53:22.578015 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46223cd0-790e-4aa3-8a62-5d057c294705 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:22.579643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46223cd0-790e-4aa3-8a62-5d057c294705 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:22.590815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1201/2408] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:53:22.595783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec8a56-1bbd-4bbe-9e32-c8102a350942 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:22.604993 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec8a56-1bbd-4bbe-9e32-c8102a350942 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:22.609812 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45971748-7b50-43d7-af2d-3b1cf25a4ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:22.610426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971748-7b50-43d7-af2d-3b1cf25a4ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.610706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971748-7b50-43d7-af2d-3b1cf25a4ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.610976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971748-7b50-43d7-af2d-3b1cf25a4ea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.611669 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45971748-7b50-43d7-af2d-3b1cf25a4ea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:22.613550 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45971748-7b50-43d7-af2d-3b1cf25a4ea5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:22.642732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1208/2409] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:22 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:22.646686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46223cd0-790e-4aa3-8a62-5d057c294705 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:22.656747 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46223cd0-790e-4aa3-8a62-5d057c294705 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:22.711250 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae0162-7d6a-44d6-b582-f243a6fe5aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:22.711803 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae0162-7d6a-44d6-b582-f243a6fe5aeb admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.712103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae0162-7d6a-44d6-b582-f243a6fe5aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.712375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae0162-7d6a-44d6-b582-f243a6fe5aeb admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.712992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae0162-7d6a-44d6-b582-f243a6fe5aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:22.715067 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae0162-7d6a-44d6-b582-f243a6fe5aeb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:22.739070 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1202/2410] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:22 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:22.740247 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45971748-7b50-43d7-af2d-3b1cf25a4ea5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:22.751309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45971748-7b50-43d7-af2d-3b1cf25a4ea5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:22.755844 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e31bc-038c-4c64-84e9-48b9ddf54cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:22.756432 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e31bc-038c-4c64-84e9-48b9ddf54cff admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.756711 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e31bc-038c-4c64-84e9-48b9ddf54cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.756978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e31bc-038c-4c64-84e9-48b9ddf54cff admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.757604 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e31bc-038c-4c64-84e9-48b9ddf54cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:22.759223 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e31bc-038c-4c64-84e9-48b9ddf54cff admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:22.769735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1209/2411] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:22.773592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae0162-7d6a-44d6-b582-f243a6fe5aeb admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:22.783862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae0162-7d6a-44d6-b582-f243a6fe5aeb admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:22.788375 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a51f3-3c44-4b9a-bcd9-715c51fefeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:22.788963 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a51f3-3c44-4b9a-bcd9-715c51fefeba admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.789240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a51f3-3c44-4b9a-bcd9-715c51fefeba admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.789508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a51f3-3c44-4b9a-bcd9-715c51fefeba admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.790139 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a51f3-3c44-4b9a-bcd9-715c51fefeba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:22.791737 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a51f3-3c44-4b9a-bcd9-715c51fefeba admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:22.802072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1203/2412] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:22.805992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e31bc-038c-4c64-84e9-48b9ddf54cff admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:22.816135 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e31bc-038c-4c64-84e9-48b9ddf54cff admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:22.820735 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d719cc-bc29-471e-b0ff-155387b1bf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:22.821357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d719cc-bc29-471e-b0ff-155387b1bf57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.821650 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d719cc-bc29-471e-b0ff-155387b1bf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.821888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d719cc-bc29-471e-b0ff-155387b1bf57 admin admin] PATH_INFO: `/v3/projects/fb8cb8c044ad40a09528d3e1248642a2/users/d480fb668fa446debc095215f4d55928/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.838634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d719cc-bc29-471e-b0ff-155387b1bf57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb8cb8c044ad40a09528d3e1248642a2, user_id=d480fb668fa446debc095215f4d55928, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:22.840876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d719cc-bc29-471e-b0ff-155387b1bf57 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:22.855624 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1210/2413] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:22 2022] PUT /identity/v3/projects/fb8cb8c044ad40a09528d3e1248642a2/users/d480fb668fa446debc095215f4d55928/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:22.861021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b832125-f0d1-4e45-8c17-9e68c36dfd9d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.861453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b832125-f0d1-4e45-8c17-9e68c36dfd9d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.861453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b832125-f0d1-4e45-8c17-9e68c36dfd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.881741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-3b832125-f0d1-4e45-8c17-9e68c36dfd9d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:22.882967 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3b832125-f0d1-4e45-8c17-9e68c36dfd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:22.887351 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d719cc-bc29-471e-b0ff-155387b1bf57 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:22.889618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c1d719cc-bc29-471e-b0ff-155387b1bf57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:22.903371 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:22.934179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:22.948368 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1204/2414] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:53:22.958595 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d719cc-bc29-471e-b0ff-155387b1bf57 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:22.958791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b832125-f0d1-4e45-8c17-9e68c36dfd9d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:22.960052 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3b832125-f0d1-4e45-8c17-9e68c36dfd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:22.963694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:22.964299 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:22.964579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:22.964888 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] PATH_INFO: `/v3/users/eaee634ad3504c26992a8a0d1fbf7f79` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:22.966020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaee634ad3504c26992a8a0d1fbf7f79)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:22.968021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:23.015924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaee634ad3504c26992a8a0d1fbf7f79'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.016598 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaee634ad3504c26992a8a0d1fbf7f79'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.019751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b832125-f0d1-4e45-8c17-9e68c36dfd9d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:23.024973 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d775a48-dc2b-48a5-ab1e-3f97465a2787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:23.025593 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d775a48-dc2b-48a5-ab1e-3f97465a2787 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:23.026045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d775a48-dc2b-48a5-ab1e-3f97465a2787 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:23.026262 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d775a48-dc2b-48a5-ab1e-3f97465a2787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:23.029615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaee634ad3504c26992a8a0d1fbf7f79'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.030557 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaee634ad3504c26992a8a0d1fbf7f79'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.036500 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d775a48-dc2b-48a5-ab1e-3f97465a2787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:23.038149 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d775a48-dc2b-48a5-ab1e-3f97465a2787 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:23.038880 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaee634ad3504c26992a8a0d1fbf7f79'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.049547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1205/2415] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:53:23.054314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1211/2416] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:22 2022] DELETE /identity/v3/users/eaee634ad3504c26992a8a0d1fbf7f79 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:53:23.058675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e3ab6-1463-4939-941d-20dc652ab0e3 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:23.059030 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e3ab6-1463-4939-941d-20dc652ab0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:23.059371 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e3ab6-1463-4939-941d-20dc652ab0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:23.080565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-384e3ab6-1463-4939-941d-20dc652ab0e3 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:23.081852 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-384e3ab6-1463-4939-941d-20dc652ab0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:23.143934 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1206/2417] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:53:23.153981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:23.156020 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:23.224729 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dd0fa-b2d1-431f-9a6b-154b170c4395 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:23.229253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da6df1-5871-44a3-b875-42d494ad46f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:23.229788 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da6df1-5871-44a3-b875-42d494ad46f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:23.230057 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da6df1-5871-44a3-b875-42d494ad46f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:23.230862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da6df1-5871-44a3-b875-42d494ad46f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:23.241211 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da6df1-5871-44a3-b875-42d494ad46f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:23.242657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da6df1-5871-44a3-b875-42d494ad46f2 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:23.253168 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1212/2418] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:23.559860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e3ab6-1463-4939-941d-20dc652ab0e3 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:23.561334 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-384e3ab6-1463-4939-941d-20dc652ab0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:23.579403 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e3ab6-1463-4939-941d-20dc652ab0e3 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:23.586945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:23.587545 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:23.587884 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:23.588120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] PATH_INFO: `/v3/projects/11922c4b1f9349dba8a7ed22ee7c3d0e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:23.589210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11922c4b1f9349dba8a7ed22ee7c3d0e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:23.591189 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:23.642917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11922c4b1f9349dba8a7ed22ee7c3d0e', 'request_id': 'req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.651701 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11922c4b1f9349dba8a7ed22ee7c3d0e', 'request_id': 'req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.659862 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11922c4b1f9349dba8a7ed22ee7c3d0e', 'request_id': 'req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.670902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] The token cache is being invalidate because project 11922c4b1f9349dba8a7ed22ee7c3d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:23.672116 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.673003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.675013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1207/2419] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:23 2022] DELETE /identity/v3/projects/11922c4b1f9349dba8a7ed22ee7c3d0e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:53:23.678933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da6df1-5871-44a3-b875-42d494ad46f2 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:23.689000 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da6df1-5871-44a3-b875-42d494ad46f2 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:23.738478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:23.782378 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:23.783447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:23.783747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:23.784018 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] PATH_INFO: `/v3/users/f891964c03394f739b174303f52c1851` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:23.785302 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f891964c03394f739b174303f52c1851)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:23.787727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:23.848670 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f891964c03394f739b174303f52c1851'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.850014 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f891964c03394f739b174303f52c1851'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.865478 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f891964c03394f739b174303f52c1851'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.867586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f891964c03394f739b174303f52c1851'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.877023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f891964c03394f739b174303f52c1851'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:23.892673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1213/2420] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:23 2022] DELETE /identity/v3/users/f891964c03394f739b174303f52c1851 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:24.188248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:24.192091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:24.215496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994bf8c-eb9f-4c58-97bc-760d3de6cdf8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:24.237604 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:24.303245 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:24.303894 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:24.304751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:24.305026 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] PATH_INFO: `/v3/projects/e3d0a0646f604bec80871972d81c53dd` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:24.306197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d0a0646f604bec80871972d81c53dd)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:24.308103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:24.364429 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d0a0646f604bec80871972d81c53dd', 'request_id': 'req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:24.375239 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d0a0646f604bec80871972d81c53dd', 'request_id': 'req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:24.383517 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d0a0646f604bec80871972d81c53dd', 'request_id': 'req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:24.396205 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] The token cache is being invalidate because project e3d0a0646f604bec80871972d81c53dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:24.396495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:24.397183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:24.398929 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1208/2421] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:24 2022] DELETE /identity/v3/projects/e3d0a0646f604bec80871972d81c53dd => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:24.402914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:24.404883 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:24.420277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58507a0c-4559-4ecc-a3f6-42cb6edafe9a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:24.488581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d81d0f-1ac2-4360-a2ec-7ac0d8b906e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:24.489214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d81d0f-1ac2-4360-a2ec-7ac0d8b906e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:24.489933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d81d0f-1ac2-4360-a2ec-7ac0d8b906e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:24.489933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d81d0f-1ac2-4360-a2ec-7ac0d8b906e5 admin admin] PATH_INFO: `/v3/users/f3e02552782a48da8e94e8a8ee0b5eb6` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:24.490959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d81d0f-1ac2-4360-a2ec-7ac0d8b906e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e02552782a48da8e94e8a8ee0b5eb6)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:24.492981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d81d0f-1ac2-4360-a2ec-7ac0d8b906e5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:24.572239 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-12d81d0f-1ac2-4360-a2ec-7ac0d8b906e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e02552782a48da8e94e8a8ee0b5eb6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:24.572957 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-12d81d0f-1ac2-4360-a2ec-7ac0d8b906e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e02552782a48da8e94e8a8ee0b5eb6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:24.579076 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:24.588047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-12d81d0f-1ac2-4360-a2ec-7ac0d8b906e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e02552782a48da8e94e8a8ee0b5eb6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:24.589526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-12d81d0f-1ac2-4360-a2ec-7ac0d8b906e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e02552782a48da8e94e8a8ee0b5eb6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:24.598703 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-12d81d0f-1ac2-4360-a2ec-7ac0d8b906e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e02552782a48da8e94e8a8ee0b5eb6'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:24.621255 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1214/2422] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:24 2022] DELETE /identity/v3/users/f3e02552782a48da8e94e8a8ee0b5eb6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:24.955992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:24.958396 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:24.979820 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd6e68-b9a3-4b9d-b703-3ec384731a8d admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:25.046654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:25.047835 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:25.048478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:25.048757 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] PATH_INFO: `/v3/projects/f83db658b5494a01bb535d15bc3e8299` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:25.049935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f83db658b5494a01bb535d15bc3e8299)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:25.051902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:25.114156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f83db658b5494a01bb535d15bc3e8299', 'request_id': 'req-34df3a56-99ba-403d-bc6e-78ab9563c2ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:25.117074 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:25.130229 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f83db658b5494a01bb535d15bc3e8299', 'request_id': 'req-34df3a56-99ba-403d-bc6e-78ab9563c2ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:25.139196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f83db658b5494a01bb535d15bc3e8299', 'request_id': 'req-34df3a56-99ba-403d-bc6e-78ab9563c2ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:25.157944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] The token cache is being invalidate because project f83db658b5494a01bb535d15bc3e8299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:25.158403 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:25.159096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-34df3a56-99ba-403d-bc6e-78ab9563c2ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:25.160609 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1209/2423] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:24 2022] DELETE /identity/v3/projects/f83db658b5494a01bb535d15bc3e8299 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:25.165638 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94994373-2c93-49d1-9b1d-faab6de6629c None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:25.166541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94994373-2c93-49d1-9b1d-faab6de6629c None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:25.166541 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94994373-2c93-49d1-9b1d-faab6de6629c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:25.191984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-94994373-2c93-49d1-9b1d-faab6de6629c None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:25.193509 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-94994373-2c93-49d1-9b1d-faab6de6629c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:25.287829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1215/2424] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:53:25.295783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1f7ad-4501-4b64-b5f4-87a62a05b49e None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:25.296782 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1f7ad-4501-4b64-b5f4-87a62a05b49e None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:25.296962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1f7ad-4501-4b64-b5f4-87a62a05b49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:25.323071 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-05a1f7ad-4501-4b64-b5f4-87a62a05b49e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:25.324690 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-05a1f7ad-4501-4b64-b5f4-87a62a05b49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:25.335217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1210/2425] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:25.340465 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94994373-2c93-49d1-9b1d-faab6de6629c None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:25.355120 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94994373-2c93-49d1-9b1d-faab6de6629c None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:25.361675 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb484b2c-95f7-4c1b-ad35-a74c4f832bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:25.362354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb484b2c-95f7-4c1b-ad35-a74c4f832bbf admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:25.362678 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb484b2c-95f7-4c1b-ad35-a74c4f832bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:25.363708 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb484b2c-95f7-4c1b-ad35-a74c4f832bbf admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:25.364372 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb484b2c-95f7-4c1b-ad35-a74c4f832bbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:53:25.364674 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb484b2c-95f7-4c1b-ad35-a74c4f832bbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:25.366440 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb484b2c-95f7-4c1b-ad35-a74c4f832bbf admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:25.384472 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1216/2426] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:53:25.385686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1f7ad-4501-4b64-b5f4-87a62a05b49e None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:25.395815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1f7ad-4501-4b64-b5f4-87a62a05b49e None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:25.403379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d832ce-aaaa-4e0e-95f6-b9ae3772911b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:25.404018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d832ce-aaaa-4e0e-95f6-b9ae3772911b admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:25.404300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d832ce-aaaa-4e0e-95f6-b9ae3772911b admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:25.404878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d832ce-aaaa-4e0e-95f6-b9ae3772911b admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:25.405867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d832ce-aaaa-4e0e-95f6-b9ae3772911b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:25.408039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d832ce-aaaa-4e0e-95f6-b9ae3772911b admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:25.442231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1211/2427] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:25 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:25.445286 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb484b2c-95f7-4c1b-ad35-a74c4f832bbf admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:25.456528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb484b2c-95f7-4c1b-ad35-a74c4f832bbf admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:25.526985 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058f683-df8a-4e7d-a602-e89f15ca5947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:25.527948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058f683-df8a-4e7d-a602-e89f15ca5947 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:25.528153 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058f683-df8a-4e7d-a602-e89f15ca5947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:25.528424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058f683-df8a-4e7d-a602-e89f15ca5947 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:25.529056 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058f683-df8a-4e7d-a602-e89f15ca5947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:25.531531 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058f683-df8a-4e7d-a602-e89f15ca5947 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:25.569644 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1217/2428] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:25 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:53:25.570970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d832ce-aaaa-4e0e-95f6-b9ae3772911b admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:25.584216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d832ce-aaaa-4e0e-95f6-b9ae3772911b admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:25.590391 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35611b2-52e3-4374-a65f-235e70c7129a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:25.591497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35611b2-52e3-4374-a65f-235e70c7129a admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:25.591846 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35611b2-52e3-4374-a65f-235e70c7129a admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:25.592182 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35611b2-52e3-4374-a65f-235e70c7129a admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:25.594055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35611b2-52e3-4374-a65f-235e70c7129a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:25.596372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35611b2-52e3-4374-a65f-235e70c7129a admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:25.631664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1212/2429] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:25.633305 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058f683-df8a-4e7d-a602-e89f15ca5947 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:25.651267 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058f683-df8a-4e7d-a602-e89f15ca5947 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:25.656784 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552c8d4-4242-4d23-8ec4-ae0a13cf0981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:25.657647 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552c8d4-4242-4d23-8ec4-ae0a13cf0981 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:25.657996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552c8d4-4242-4d23-8ec4-ae0a13cf0981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:25.658373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552c8d4-4242-4d23-8ec4-ae0a13cf0981 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:25.659175 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552c8d4-4242-4d23-8ec4-ae0a13cf0981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:25.660858 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552c8d4-4242-4d23-8ec4-ae0a13cf0981 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:25.673250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1218/2430] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:25.681232 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35611b2-52e3-4374-a65f-235e70c7129a admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:25.694362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35611b2-52e3-4374-a65f-235e70c7129a admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:25.699850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d8fc0-736f-44df-a15b-20ec23774152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:25.700687 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d8fc0-736f-44df-a15b-20ec23774152 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:25.700974 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d8fc0-736f-44df-a15b-20ec23774152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:25.701417 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d8fc0-736f-44df-a15b-20ec23774152 admin admin] PATH_INFO: `/v3/projects/a44c1dd3ec4d4da782ac8eba75e210d3/users/afb191f4ac574cb5bc5de28da757a258/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:25.717746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d8fc0-736f-44df-a15b-20ec23774152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44c1dd3ec4d4da782ac8eba75e210d3, user_id=afb191f4ac574cb5bc5de28da757a258, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:25.721018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d8fc0-736f-44df-a15b-20ec23774152 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:25.746569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1213/2431] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:25 2022] PUT /identity/v3/projects/a44c1dd3ec4d4da782ac8eba75e210d3/users/afb191f4ac574cb5bc5de28da757a258/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:25.754560 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ce5b6-a460-42b4-adb8-1232978c2806 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:25.754852 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ce5b6-a460-42b4-adb8-1232978c2806 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:25.755129 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ce5b6-a460-42b4-adb8-1232978c2806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:25.789250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f96ce5b6-a460-42b4-adb8-1232978c2806 None None] MFA Rules not processed for user `afb191f4ac574cb5bc5de28da757a258`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:25.790578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f96ce5b6-a460-42b4-adb8-1232978c2806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:25.832573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:25.899830 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1219/2432] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:25.912304 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d8fc0-736f-44df-a15b-20ec23774152 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:25.914556 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-be8d8fc0-736f-44df-a15b-20ec23774152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:25.987876 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d8fc0-736f-44df-a15b-20ec23774152 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:25.992412 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb6149-5561-4f17-9446-9d36cae90ddb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:25.992909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb6149-5561-4f17-9446-9d36cae90ddb service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:25.993261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb6149-5561-4f17-9446-9d36cae90ddb service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:25.993611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb6149-5561-4f17-9446-9d36cae90ddb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:26.003240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb6149-5561-4f17-9446-9d36cae90ddb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:26.004775 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb6149-5561-4f17-9446-9d36cae90ddb service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:26.014601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1214/2433] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:27.340736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb6d60-d0d5-4273-846c-d9c6c7e4f05a None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:27.341172 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb6d60-d0d5-4273-846c-d9c6c7e4f05a None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:27.341272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb6d60-d0d5-4273-846c-d9c6c7e4f05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:27.370265 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-64eb6d60-d0d5-4273-846c-d9c6c7e4f05a None None] MFA Rules not processed for user `d480fb668fa446debc095215f4d55928`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:27.371401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-64eb6d60-d0d5-4273-846c-d9c6c7e4f05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:27.457614 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1220/2434] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:27.462128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb6149-5561-4f17-9446-9d36cae90ddb service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:27.464141 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-70cb6149-5561-4f17-9446-9d36cae90ddb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:27.539475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb6149-5561-4f17-9446-9d36cae90ddb service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:27.544051 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8441d2-8921-4d18-a0dd-8d591ef11ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:27.546953 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8441d2-8921-4d18-a0dd-8d591ef11ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:27.547246 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8441d2-8921-4d18-a0dd-8d591ef11ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:27.547531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8441d2-8921-4d18-a0dd-8d591ef11ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:27.548179 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8441d2-8921-4d18-a0dd-8d591ef11ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:27.550126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8441d2-8921-4d18-a0dd-8d591ef11ec7 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:27.610302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1215/2435] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:27 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:27.614147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb6d60-d0d5-4273-846c-d9c6c7e4f05a None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:27.625992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb6d60-d0d5-4273-846c-d9c6c7e4f05a None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:27.699609 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06df836-cb17-4aa3-a280-346176a39697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:27.700282 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06df836-cb17-4aa3-a280-346176a39697 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:27.700561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06df836-cb17-4aa3-a280-346176a39697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:27.700831 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06df836-cb17-4aa3-a280-346176a39697 admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:27.701480 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06df836-cb17-4aa3-a280-346176a39697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:27.704021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06df836-cb17-4aa3-a280-346176a39697 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:27.738570 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1221/2436] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:27 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:27.739634 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8441d2-8921-4d18-a0dd-8d591ef11ec7 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:27.750133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:27.756565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8441d2-8921-4d18-a0dd-8d591ef11ec7 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:27.763359 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db105e9-f6e3-4165-90a9-b22bdbcf19fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:27.763979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db105e9-f6e3-4165-90a9-b22bdbcf19fa admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:27.764404 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db105e9-f6e3-4165-90a9-b22bdbcf19fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:27.764496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db105e9-f6e3-4165-90a9-b22bdbcf19fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:27.766898 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db105e9-f6e3-4165-90a9-b22bdbcf19fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:27.768564 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db105e9-f6e3-4165-90a9-b22bdbcf19fa admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:27.780089 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1216/2437] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:27.785921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06df836-cb17-4aa3-a280-346176a39697 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:27.798736 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06df836-cb17-4aa3-a280-346176a39697 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:27.804060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0df37-44b9-4694-875a-d188e103d764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:27.804838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0df37-44b9-4694-875a-d188e103d764 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:27.805181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0df37-44b9-4694-875a-d188e103d764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:27.805524 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0df37-44b9-4694-875a-d188e103d764 admin admin] PATH_INFO: `/v3/projects/f912c154514849c6af15ecb9fecd1907/users/c76b09da40e04e36a993ca4239055708/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:27.825258 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0df37-44b9-4694-875a-d188e103d764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f912c154514849c6af15ecb9fecd1907, user_id=c76b09da40e04e36a993ca4239055708, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:27.828081 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0df37-44b9-4694-875a-d188e103d764 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:27.860925 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1222/2438] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:27 2022] PUT /identity/v3/projects/f912c154514849c6af15ecb9fecd1907/users/c76b09da40e04e36a993ca4239055708/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:27.864721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db105e9-f6e3-4165-90a9-b22bdbcf19fa admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:27.876156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db105e9-f6e3-4165-90a9-b22bdbcf19fa admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:27.955466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfde45-d34d-4794-b250-088f27c74dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:27.956081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfde45-d34d-4794-b250-088f27c74dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:27.956361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfde45-d34d-4794-b250-088f27c74dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:27.956632 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfde45-d34d-4794-b250-088f27c74dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:27.957408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfde45-d34d-4794-b250-088f27c74dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:27.959153 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfde45-d34d-4794-b250-088f27c74dfb admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:27.973174 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1217/2439] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:27.974396 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0df37-44b9-4694-875a-d188e103d764 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:27.987087 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0df37-44b9-4694-875a-d188e103d764 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:27.992036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1a7a4-a759-47fe-9a0e-61ff98613360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:27.994112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1a7a4-a759-47fe-9a0e-61ff98613360 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:27.994112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1a7a4-a759-47fe-9a0e-61ff98613360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:27.994112 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1a7a4-a759-47fe-9a0e-61ff98613360 admin admin] PATH_INFO: `/v3/projects/f912c154514849c6af15ecb9fecd1907/users/c76b09da40e04e36a993ca4239055708/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:27.995435 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1a7a4-a759-47fe-9a0e-61ff98613360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f912c154514849c6af15ecb9fecd1907, user_id=c76b09da40e04e36a993ca4239055708, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:27.998044 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1a7a4-a759-47fe-9a0e-61ff98613360 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:28.009583 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1223/2440] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:27 2022] PUT /identity/v3/projects/f912c154514849c6af15ecb9fecd1907/users/c76b09da40e04e36a993ca4239055708/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:33.095658 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6a6a4-b5bb-4134-a52d-aa2a4607ef45 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:33.096208 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6a6a4-b5bb-4134-a52d-aa2a4607ef45 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:33.096208 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6a6a4-b5bb-4134-a52d-aa2a4607ef45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:33.138296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-97c6a6a4-b5bb-4134-a52d-aa2a4607ef45 None None] MFA Rules not processed for user `c76b09da40e04e36a993ca4239055708`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:33.141925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-97c6a6a4-b5bb-4134-a52d-aa2a4607ef45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:33.260035 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1218/2441] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:33.275704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1a7a4-a759-47fe-9a0e-61ff98613360 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:33.277987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6ce1a7a4-a759-47fe-9a0e-61ff98613360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:33.294959 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:33.383684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1a7a4-a759-47fe-9a0e-61ff98613360 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:33.389333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3008475-9807-4a19-847d-2d2afcb9300e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:33.390517 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3008475-9807-4a19-847d-2d2afcb9300e service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:33.390794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3008475-9807-4a19-847d-2d2afcb9300e service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:33.391060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3008475-9807-4a19-847d-2d2afcb9300e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:33.455548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3008475-9807-4a19-847d-2d2afcb9300e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:33.457182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3008475-9807-4a19-847d-2d2afcb9300e service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:33.513574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1224/2442] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:35.660401 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85026d58-e957-40bd-9c03-e3b4cbadd8f7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:35.660903 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85026d58-e957-40bd-9c03-e3b4cbadd8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:35.661000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85026d58-e957-40bd-9c03-e3b4cbadd8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:35.680631 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-85026d58-e957-40bd-9c03-e3b4cbadd8f7 None None] MFA Rules not processed for user `d480fb668fa446debc095215f4d55928`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:35.681746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-85026d58-e957-40bd-9c03-e3b4cbadd8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:35.689533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1219/2443] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:37.112750 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3008475-9807-4a19-847d-2d2afcb9300e service nova] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:37.250542 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3008475-9807-4a19-847d-2d2afcb9300e service nova] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:37.261067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:37.262661 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:37.263392 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:37.263908 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] PATH_INFO: `/v3/users/afb191f4ac574cb5bc5de28da757a258` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:37.274700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb191f4ac574cb5bc5de28da757a258)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:37.281189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:37.372123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb191f4ac574cb5bc5de28da757a258'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:37.372918 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb191f4ac574cb5bc5de28da757a258'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:37.375578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:37.407040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb191f4ac574cb5bc5de28da757a258'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:37.408029 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb191f4ac574cb5bc5de28da757a258'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:37.418914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb191f4ac574cb5bc5de28da757a258'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:37.444129 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1225/2444] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:37 2022] DELETE /identity/v3/users/afb191f4ac574cb5bc5de28da757a258 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:37.448605 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398976cd-af44-42f9-a653-d0fce6255d02 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:37.449165 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398976cd-af44-42f9-a653-d0fce6255d02 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:37.449165 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398976cd-af44-42f9-a653-d0fce6255d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:37.470861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-398976cd-af44-42f9-a653-d0fce6255d02 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:37.472637 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-398976cd-af44-42f9-a653-d0fce6255d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:37.567816 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1220/2445] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:37.580729 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:37.582131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:37.680130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be621067-af76-4498-b8a4-df5c8ba1fbf3 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:37.694877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebeaa5-ed24-4682-9907-53f00af45d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:37.695791 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebeaa5-ed24-4682-9907-53f00af45d14 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:37.695924 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebeaa5-ed24-4682-9907-53f00af45d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:37.696266 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebeaa5-ed24-4682-9907-53f00af45d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:37.722130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eebeaa5-ed24-4682-9907-53f00af45d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:37.724283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eebeaa5-ed24-4682-9907-53f00af45d14 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:37.749686 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1226/2446] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:38.140126 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398976cd-af44-42f9-a653-d0fce6255d02 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:38.141660 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-398976cd-af44-42f9-a653-d0fce6255d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:38.158566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398976cd-af44-42f9-a653-d0fce6255d02 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:38.164263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:38.165283 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:38.165918 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:38.166469 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] PATH_INFO: `/v3/projects/a44c1dd3ec4d4da782ac8eba75e210d3` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:38.168200 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a44c1dd3ec4d4da782ac8eba75e210d3)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:38.171439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:38.174037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:38.245534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a44c1dd3ec4d4da782ac8eba75e210d3', 'request_id': 'req-1652aab5-af69-4c63-8324-ddc5831bc6a3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:38.261880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a44c1dd3ec4d4da782ac8eba75e210d3', 'request_id': 'req-1652aab5-af69-4c63-8324-ddc5831bc6a3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:38.270882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a44c1dd3ec4d4da782ac8eba75e210d3', 'request_id': 'req-1652aab5-af69-4c63-8324-ddc5831bc6a3'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:38.286957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] The token cache is being invalidate because project a44c1dd3ec4d4da782ac8eba75e210d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:38.290685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:38.292917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:38.298259 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1221/2447] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:38 2022] DELETE /identity/v3/projects/a44c1dd3ec4d4da782ac8eba75e210d3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:42.045475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebeaa5-ed24-4682-9907-53f00af45d14 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:42.047132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5eebeaa5-ed24-4682-9907-53f00af45d14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:42.058290 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebeaa5-ed24-4682-9907-53f00af45d14 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:42.107845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba206067-3797-4b5b-8cba-9d3cf43a5012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:42.108536 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba206067-3797-4b5b-8cba-9d3cf43a5012 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:42.108817 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba206067-3797-4b5b-8cba-9d3cf43a5012 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:42.109093 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba206067-3797-4b5b-8cba-9d3cf43a5012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:42.110958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-ba206067-3797-4b5b-8cba-9d3cf43a5012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:42.173783 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba206067-3797-4b5b-8cba-9d3cf43a5012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:42.175250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba206067-3797-4b5b-8cba-9d3cf43a5012 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:42.185845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1227/2448] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:45.481124 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:45.482870 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:45.510009 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652aab5-af69-4c63-8324-ddc5831bc6a3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:45.516207 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355f31e-7c43-407a-bc56-ba225ab96303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:45.517050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355f31e-7c43-407a-bc56-ba225ab96303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:45.517338 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355f31e-7c43-407a-bc56-ba225ab96303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:45.517868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355f31e-7c43-407a-bc56-ba225ab96303 admin admin] PATH_INFO: `/v3/users/16228db2ddd147969482871b600ac797` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:45.539217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355f31e-7c43-407a-bc56-ba225ab96303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16228db2ddd147969482871b600ac797)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:45.541347 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355f31e-7c43-407a-bc56-ba225ab96303 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:45.593906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a355f31e-7c43-407a-bc56-ba225ab96303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16228db2ddd147969482871b600ac797'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:45.594734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a355f31e-7c43-407a-bc56-ba225ab96303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16228db2ddd147969482871b600ac797'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:45.595335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a355f31e-7c43-407a-bc56-ba225ab96303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16228db2ddd147969482871b600ac797'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:45.602478 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a355f31e-7c43-407a-bc56-ba225ab96303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16228db2ddd147969482871b600ac797'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:45.616032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a355f31e-7c43-407a-bc56-ba225ab96303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16228db2ddd147969482871b600ac797'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:45.630502 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1222/2449] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:45 2022] DELETE /identity/v3/users/16228db2ddd147969482871b600ac797 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:45.810445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba206067-3797-4b5b-8cba-9d3cf43a5012 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:45.820864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba206067-3797-4b5b-8cba-9d3cf43a5012 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:45.853692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:45.900036 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:45.900663 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:45.900945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:45.901218 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] PATH_INFO: `/v3/projects/a578ee882e60456392c88a508d1a6db3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:45.911653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a578ee882e60456392c88a508d1a6db3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:45.913536 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:45.959789 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a578ee882e60456392c88a508d1a6db3', 'request_id': 'req-872da65a-07f4-441e-8ce1-5e831b539dae'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:45.965383 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a578ee882e60456392c88a508d1a6db3', 'request_id': 'req-872da65a-07f4-441e-8ce1-5e831b539dae'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:45.972663 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a578ee882e60456392c88a508d1a6db3', 'request_id': 'req-872da65a-07f4-441e-8ce1-5e831b539dae'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:45.984801 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] The token cache is being invalidate because project a578ee882e60456392c88a508d1a6db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:45.985022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:45.985657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-872da65a-07f4-441e-8ce1-5e831b539dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:45.987157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1228/2450] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:45 2022] DELETE /identity/v3/projects/a578ee882e60456392c88a508d1a6db3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:45.991997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a900e-0a51-48fa-9670-7df5f028d204 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:45.992315 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a900e-0a51-48fa-9670-7df5f028d204 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:45.992592 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a900e-0a51-48fa-9670-7df5f028d204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.013317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f39a900e-0a51-48fa-9670-7df5f028d204 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:46.015247 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f39a900e-0a51-48fa-9670-7df5f028d204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:46.064239 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:46.079279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1223/2451] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:53:46.087903 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ac03-dafc-450f-a0bf-1ac538d747a8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.088189 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ac03-dafc-450f-a0bf-1ac538d747a8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.088452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ac03-dafc-450f-a0bf-1ac538d747a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.108207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-3ff9ac03-dafc-450f-a0bf-1ac538d747a8 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:46.109445 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-3ff9ac03-dafc-450f-a0bf-1ac538d747a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:46.117287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1229/2452] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:46.121457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a900e-0a51-48fa-9670-7df5f028d204 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.131631 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a900e-0a51-48fa-9670-7df5f028d204 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.136296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47fb04-711b-4d00-af0a-c48766da1a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.136893 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47fb04-711b-4d00-af0a-c48766da1a7f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.137163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47fb04-711b-4d00-af0a-c48766da1a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.137429 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47fb04-711b-4d00-af0a-c48766da1a7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.138093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47fb04-711b-4d00-af0a-c48766da1a7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:53:46.138493 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47fb04-711b-4d00-af0a-c48766da1a7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.140677 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47fb04-711b-4d00-af0a-c48766da1a7f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.151985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1224/2453] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:53:46.156377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ac03-dafc-450f-a0bf-1ac538d747a8 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.165723 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ac03-dafc-450f-a0bf-1ac538d747a8 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.170381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8f561-6d6e-4aa4-875d-ca490301efae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.171193 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8f561-6d6e-4aa4-875d-ca490301efae admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.171470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8f561-6d6e-4aa4-875d-ca490301efae admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.171738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8f561-6d6e-4aa4-875d-ca490301efae admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.172358 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8f561-6d6e-4aa4-875d-ca490301efae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.174294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8f561-6d6e-4aa4-875d-ca490301efae admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.204368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1230/2454] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:46 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:46.208699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47fb04-711b-4d00-af0a-c48766da1a7f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.219561 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47fb04-711b-4d00-af0a-c48766da1a7f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.275130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728152e5-a61c-48ca-a9cd-c1fd31fe87a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.275742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728152e5-a61c-48ca-a9cd-c1fd31fe87a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.276026 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728152e5-a61c-48ca-a9cd-c1fd31fe87a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.276300 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728152e5-a61c-48ca-a9cd-c1fd31fe87a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.276933 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728152e5-a61c-48ca-a9cd-c1fd31fe87a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.278878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728152e5-a61c-48ca-a9cd-c1fd31fe87a5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.303036 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1225/2455] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:46 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:46.307170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8f561-6d6e-4aa4-875d-ca490301efae admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.318016 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8f561-6d6e-4aa4-875d-ca490301efae admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.322670 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475399f-34fd-4cec-bf0c-b11fa6931522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.323256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475399f-34fd-4cec-bf0c-b11fa6931522 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.323535 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475399f-34fd-4cec-bf0c-b11fa6931522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.323806 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475399f-34fd-4cec-bf0c-b11fa6931522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.324441 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475399f-34fd-4cec-bf0c-b11fa6931522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.326009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475399f-34fd-4cec-bf0c-b11fa6931522 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.336149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1231/2456] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:46.340107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728152e5-a61c-48ca-a9cd-c1fd31fe87a5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.350133 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728152e5-a61c-48ca-a9cd-c1fd31fe87a5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.354670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37649886-35d2-4b21-b742-c0f70b5a2abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.355254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37649886-35d2-4b21-b742-c0f70b5a2abc admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.355529 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37649886-35d2-4b21-b742-c0f70b5a2abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.355795 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37649886-35d2-4b21-b742-c0f70b5a2abc admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.356423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37649886-35d2-4b21-b742-c0f70b5a2abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.357988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37649886-35d2-4b21-b742-c0f70b5a2abc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.368891 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1226/2457] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:46.373163 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475399f-34fd-4cec-bf0c-b11fa6931522 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.383707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475399f-34fd-4cec-bf0c-b11fa6931522 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.388318 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f84036-6b62-4a54-b902-7c046a5f9052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.388956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f84036-6b62-4a54-b902-7c046a5f9052 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.389234 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f84036-6b62-4a54-b902-7c046a5f9052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.389577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f84036-6b62-4a54-b902-7c046a5f9052 admin admin] PATH_INFO: `/v3/projects/4aed8c29410044e4bd7186e03bb54fe0/users/35413e2182344ae69cc08a912e9fb2c8/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.404151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f84036-6b62-4a54-b902-7c046a5f9052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aed8c29410044e4bd7186e03bb54fe0, user_id=35413e2182344ae69cc08a912e9fb2c8, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.407365 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f84036-6b62-4a54-b902-7c046a5f9052 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.419922 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1232/2458] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:46 2022] PUT /identity/v3/projects/4aed8c29410044e4bd7186e03bb54fe0/users/35413e2182344ae69cc08a912e9fb2c8/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:53:46.431164 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9b4e5-3e13-4df2-b4ae-585b5077a598 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.431444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9b4e5-3e13-4df2-b4ae-585b5077a598 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.431713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9b4e5-3e13-4df2-b4ae-585b5077a598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.459459 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-91a9b4e5-3e13-4df2-b4ae-585b5077a598 None None] MFA Rules not processed for user `35413e2182344ae69cc08a912e9fb2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:46.460925 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-91a9b4e5-3e13-4df2-b4ae-585b5077a598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:46.525111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1227/2459] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:46.529490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f84036-6b62-4a54-b902-7c046a5f9052 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.568758 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:46.592743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f84036-6b62-4a54-b902-7c046a5f9052 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.597367 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5798484-57e9-4ca5-9c85-c55fd5b02292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.597972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5798484-57e9-4ca5-9c85-c55fd5b02292 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.598278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5798484-57e9-4ca5-9c85-c55fd5b02292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.598548 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5798484-57e9-4ca5-9c85-c55fd5b02292 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.599169 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5798484-57e9-4ca5-9c85-c55fd5b02292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.601042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5798484-57e9-4ca5-9c85-c55fd5b02292 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.627818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1233/2460] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:46 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:46.632000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9b4e5-3e13-4df2-b4ae-585b5077a598 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.642664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9b4e5-3e13-4df2-b4ae-585b5077a598 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.689360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:46.702894 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67693be7-0f94-4f13-b58c-bb30d8e3afe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.703533 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67693be7-0f94-4f13-b58c-bb30d8e3afe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.703813 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67693be7-0f94-4f13-b58c-bb30d8e3afe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.704088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67693be7-0f94-4f13-b58c-bb30d8e3afe0 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.704730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67693be7-0f94-4f13-b58c-bb30d8e3afe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.706662 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67693be7-0f94-4f13-b58c-bb30d8e3afe0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.728661 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1228/2461] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:46 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:46.732715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5798484-57e9-4ca5-9c85-c55fd5b02292 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.743499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5798484-57e9-4ca5-9c85-c55fd5b02292 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.748090 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780047d-f53a-4586-9f39-6c6661b0ef5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.748680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780047d-f53a-4586-9f39-6c6661b0ef5f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.749111 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780047d-f53a-4586-9f39-6c6661b0ef5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.749390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780047d-f53a-4586-9f39-6c6661b0ef5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.750027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780047d-f53a-4586-9f39-6c6661b0ef5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.751653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780047d-f53a-4586-9f39-6c6661b0ef5f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.762084 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1234/2462] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:46.766151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67693be7-0f94-4f13-b58c-bb30d8e3afe0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.776101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67693be7-0f94-4f13-b58c-bb30d8e3afe0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.781305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a21b9-c56d-4834-ac65-522e7527ff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.782016 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a21b9-c56d-4834-ac65-522e7527ff32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.782348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a21b9-c56d-4834-ac65-522e7527ff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.782618 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a21b9-c56d-4834-ac65-522e7527ff32 admin admin] PATH_INFO: `/v3/projects/006fe6244c03402abbb6429ca4448101/users/fe689e10266d4a42948dba3e1dcf610d/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.797669 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a21b9-c56d-4834-ac65-522e7527ff32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=006fe6244c03402abbb6429ca4448101, user_id=fe689e10266d4a42948dba3e1dcf610d, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.800336 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a21b9-c56d-4834-ac65-522e7527ff32 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.813439 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1229/2463] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:46 2022] PUT /identity/v3/projects/006fe6244c03402abbb6429ca4448101/users/fe689e10266d4a42948dba3e1dcf610d/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:46.817437 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780047d-f53a-4586-9f39-6c6661b0ef5f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.827413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780047d-f53a-4586-9f39-6c6661b0ef5f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.884972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518f5c7-84f9-4b74-9685-379f483e61ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.885581 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518f5c7-84f9-4b74-9685-379f483e61ac admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.885860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518f5c7-84f9-4b74-9685-379f483e61ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.886131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518f5c7-84f9-4b74-9685-379f483e61ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.886798 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518f5c7-84f9-4b74-9685-379f483e61ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.888359 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518f5c7-84f9-4b74-9685-379f483e61ac admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.898423 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1235/2464] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:46.902423 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a21b9-c56d-4834-ac65-522e7527ff32 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:46.913442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a21b9-c56d-4834-ac65-522e7527ff32 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:46.918224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbd0ea-7667-414e-b556-7568346471e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:46.918874 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd0ea-7667-414e-b556-7568346471e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.919150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd0ea-7667-414e-b556-7568346471e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.919417 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd0ea-7667-414e-b556-7568346471e5 admin admin] PATH_INFO: `/v3/projects/006fe6244c03402abbb6429ca4448101/users/fe689e10266d4a42948dba3e1dcf610d/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.921994 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbd0ea-7667-414e-b556-7568346471e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=006fe6244c03402abbb6429ca4448101, user_id=fe689e10266d4a42948dba3e1dcf610d, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:46.924650 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbd0ea-7667-414e-b556-7568346471e5 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:46.934996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1230/2465] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:46 2022] PUT /identity/v3/projects/006fe6244c03402abbb6429ca4448101/users/fe689e10266d4a42948dba3e1dcf610d/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:46.946488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f8a47-d4a0-4c46-8d3e-544c480de147 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:46.946776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f8a47-d4a0-4c46-8d3e-544c480de147 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:46.947058 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f8a47-d4a0-4c46-8d3e-544c480de147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:46.974642 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7e6f8a47-d4a0-4c46-8d3e-544c480de147 None None] MFA Rules not processed for user `fe689e10266d4a42948dba3e1dcf610d`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:46.975833 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7e6f8a47-d4a0-4c46-8d3e-544c480de147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:47.046375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1236/2466] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:47.057433 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbd0ea-7667-414e-b556-7568346471e5 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:47.059303 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-34dbd0ea-7667-414e-b556-7568346471e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:47.117306 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbd0ea-7667-414e-b556-7568346471e5 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:47.120982 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95e0b6-b2dc-472a-834d-4a942feec0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:47.121579 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95e0b6-b2dc-472a-834d-4a942feec0dd service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:47.121857 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95e0b6-b2dc-472a-834d-4a942feec0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:47.122154 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95e0b6-b2dc-472a-834d-4a942feec0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:47.131288 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95e0b6-b2dc-472a-834d-4a942feec0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:47.132760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95e0b6-b2dc-472a-834d-4a942feec0dd service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:47.142769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1231/2467] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:48.474608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f8a47-d4a0-4c46-8d3e-544c480de147 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:48.507958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:48.539434 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f8a47-d4a0-4c46-8d3e-544c480de147 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:48.543834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:48.544449 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:48.544725 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:48.544989 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] PATH_INFO: `/v3/users/35413e2182344ae69cc08a912e9fb2c8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:48.546121 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35413e2182344ae69cc08a912e9fb2c8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:48.548022 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:48.595047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35413e2182344ae69cc08a912e9fb2c8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:48.595617 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35413e2182344ae69cc08a912e9fb2c8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:48.607553 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35413e2182344ae69cc08a912e9fb2c8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:48.608243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35413e2182344ae69cc08a912e9fb2c8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:48.615293 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35413e2182344ae69cc08a912e9fb2c8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:48.627987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1237/2468] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:48 2022] DELETE /identity/v3/users/35413e2182344ae69cc08a912e9fb2c8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:48.632477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26397383-51e3-48ef-bd38-621534e2dc55 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:48.632951 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26397383-51e3-48ef-bd38-621534e2dc55 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:48.633050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26397383-51e3-48ef-bd38-621534e2dc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:48.652505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-26397383-51e3-48ef-bd38-621534e2dc55 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:48.653688 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-26397383-51e3-48ef-bd38-621534e2dc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:48.711557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1232/2469] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:48.721821 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:48.723285 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:48.779130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8844d4-046d-4aba-9486-0b87363d65e0 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:48.783384 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe2f20-ac84-4e2c-9fcb-f043b9a2b2c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:48.783968 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe2f20-ac84-4e2c-9fcb-f043b9a2b2c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:48.784241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe2f20-ac84-4e2c-9fcb-f043b9a2b2c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:48.784528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe2f20-ac84-4e2c-9fcb-f043b9a2b2c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:48.793243 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe2f20-ac84-4e2c-9fcb-f043b9a2b2c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:48.794684 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe2f20-ac84-4e2c-9fcb-f043b9a2b2c6 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:48.803893 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1238/2470] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:49.079452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26397383-51e3-48ef-bd38-621534e2dc55 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:49.080944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-26397383-51e3-48ef-bd38-621534e2dc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:49.095725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26397383-51e3-48ef-bd38-621534e2dc55 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:49.100775 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:49.101427 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:49.101704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:49.101975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] PATH_INFO: `/v3/projects/4aed8c29410044e4bd7186e03bb54fe0` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:49.103235 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aed8c29410044e4bd7186e03bb54fe0)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:49.105050 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:49.145847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:49.155169 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aed8c29410044e4bd7186e03bb54fe0', 'request_id': 'req-06df6d87-ba84-4fed-b883-fccd49869308'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.162134 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aed8c29410044e4bd7186e03bb54fe0', 'request_id': 'req-06df6d87-ba84-4fed-b883-fccd49869308'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.168629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aed8c29410044e4bd7186e03bb54fe0', 'request_id': 'req-06df6d87-ba84-4fed-b883-fccd49869308'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.179318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] The token cache is being invalidate because project 4aed8c29410044e4bd7186e03bb54fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:49.179605 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.180241 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.181718 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1233/2471] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:49 2022] DELETE /identity/v3/projects/4aed8c29410044e4bd7186e03bb54fe0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:49.185810 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe2f20-ac84-4e2c-9fcb-f043b9a2b2c6 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:49.195814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe2f20-ac84-4e2c-9fcb-f043b9a2b2c6 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:49.242636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:49.252949 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2eab59-1e5a-45c7-bd2d-7be1307c4c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:49.253572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eab59-1e5a-45c7-bd2d-7be1307c4c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:49.253862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eab59-1e5a-45c7-bd2d-7be1307c4c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:49.254123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eab59-1e5a-45c7-bd2d-7be1307c4c02 admin admin] PATH_INFO: `/v3/users/fe689e10266d4a42948dba3e1dcf610d` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:49.255254 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2eab59-1e5a-45c7-bd2d-7be1307c4c02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe689e10266d4a42948dba3e1dcf610d)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:49.257154 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2eab59-1e5a-45c7-bd2d-7be1307c4c02 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:49.309436 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fc2eab59-1e5a-45c7-bd2d-7be1307c4c02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe689e10266d4a42948dba3e1dcf610d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.310074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fc2eab59-1e5a-45c7-bd2d-7be1307c4c02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe689e10266d4a42948dba3e1dcf610d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.322620 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fc2eab59-1e5a-45c7-bd2d-7be1307c4c02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe689e10266d4a42948dba3e1dcf610d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.323395 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fc2eab59-1e5a-45c7-bd2d-7be1307c4c02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe689e10266d4a42948dba3e1dcf610d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.330569 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-fc2eab59-1e5a-45c7-bd2d-7be1307c4c02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe689e10266d4a42948dba3e1dcf610d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.343450 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1239/2472] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:53:49 2022] DELETE /identity/v3/users/fe689e10266d4a42948dba3e1dcf610d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:49.609693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:49.611174 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:49.626188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df6d87-ba84-4fed-b883-fccd49869308 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:49.683796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:49.684483 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:49.684691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:49.684947 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] PATH_INFO: `/v3/projects/006fe6244c03402abbb6429ca4448101` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:49.686045 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] RBAC: Authorizing `identity:delete_project(project_id=006fe6244c03402abbb6429ca4448101)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:49.688038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:49.734400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '006fe6244c03402abbb6429ca4448101', 'request_id': 'req-7a822f45-a754-460a-902b-22326de8462e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.741414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '006fe6244c03402abbb6429ca4448101', 'request_id': 'req-7a822f45-a754-460a-902b-22326de8462e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.747419 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '006fe6244c03402abbb6429ca4448101', 'request_id': 'req-7a822f45-a754-460a-902b-22326de8462e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.759935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] The token cache is being invalidate because project 006fe6244c03402abbb6429ca4448101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:53:49.760231 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.760892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-7a822f45-a754-460a-902b-22326de8462e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:53:49.762506 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1234/2473] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:53:49 2022] DELETE /identity/v3/projects/006fe6244c03402abbb6429ca4448101 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:49.767385 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425016ac-d4b0-4275-9441-e511522a4571 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:49.767851 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425016ac-d4b0-4275-9441-e511522a4571 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:49.767979 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425016ac-d4b0-4275-9441-e511522a4571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:49.788319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-425016ac-d4b0-4275-9441-e511522a4571 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:49.789513 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-425016ac-d4b0-4275-9441-e511522a4571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:49.847916 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1240/2474] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:49.855607 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69902aa4-4ffc-4ab2-bc84-1f7b5d29274d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:49.855892 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69902aa4-4ffc-4ab2-bc84-1f7b5d29274d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:49.856163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69902aa4-4ffc-4ab2-bc84-1f7b5d29274d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:49.876174 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-69902aa4-4ffc-4ab2-bc84-1f7b5d29274d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:49.877366 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-69902aa4-4ffc-4ab2-bc84-1f7b5d29274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:49.885174 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1235/2475] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:49.889182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425016ac-d4b0-4275-9441-e511522a4571 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:49.898695 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425016ac-d4b0-4275-9441-e511522a4571 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:49.903246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1fc8b-4184-44ef-aee6-965ae1caf89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:49.903832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1fc8b-4184-44ef-aee6-965ae1caf89f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:49.904109 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1fc8b-4184-44ef-aee6-965ae1caf89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:49.904375 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1fc8b-4184-44ef-aee6-965ae1caf89f admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:49.905039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1fc8b-4184-44ef-aee6-965ae1caf89f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:53:49.905586 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1fc8b-4184-44ef-aee6-965ae1caf89f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:49.907757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1fc8b-4184-44ef-aee6-965ae1caf89f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:49.918958 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1241/2476] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:53:49.923989 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69902aa4-4ffc-4ab2-bc84-1f7b5d29274d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:49.933096 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69902aa4-4ffc-4ab2-bc84-1f7b5d29274d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:49.937731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cd74f-7e64-407e-9eb7-7a2a54ff2b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:49.938538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cd74f-7e64-407e-9eb7-7a2a54ff2b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:49.938675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cd74f-7e64-407e-9eb7-7a2a54ff2b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:49.938940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cd74f-7e64-407e-9eb7-7a2a54ff2b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:49.939566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5cd74f-7e64-407e-9eb7-7a2a54ff2b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:49.941445 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5cd74f-7e64-407e-9eb7-7a2a54ff2b7e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:49.968172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1236/2477] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:49 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:53:49.973005 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1fc8b-4184-44ef-aee6-965ae1caf89f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:49.983361 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1fc8b-4184-44ef-aee6-965ae1caf89f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:50.043885 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959794-df8d-4d4a-aeb8-bca1e9162fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:50.044494 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959794-df8d-4d4a-aeb8-bca1e9162fff admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.044779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959794-df8d-4d4a-aeb8-bca1e9162fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.045052 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959794-df8d-4d4a-aeb8-bca1e9162fff admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.045714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85959794-df8d-4d4a-aeb8-bca1e9162fff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:50.047648 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85959794-df8d-4d4a-aeb8-bca1e9162fff admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:50.056235 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:50.082132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1242/2478] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:49 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:50.086441 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cd74f-7e64-407e-9eb7-7a2a54ff2b7e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:50.097410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cd74f-7e64-407e-9eb7-7a2a54ff2b7e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:50.102177 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82cba1-88b4-43ef-871e-c13ccf42771f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:50.102783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82cba1-88b4-43ef-871e-c13ccf42771f admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.103060 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82cba1-88b4-43ef-871e-c13ccf42771f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.103330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82cba1-88b4-43ef-871e-c13ccf42771f admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.103966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82cba1-88b4-43ef-871e-c13ccf42771f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:50.105549 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82cba1-88b4-43ef-871e-c13ccf42771f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:50.115831 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1237/2479] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:50.119748 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959794-df8d-4d4a-aeb8-bca1e9162fff admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:50.129698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959794-df8d-4d4a-aeb8-bca1e9162fff admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:50.134344 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc8a55-e9f7-46bd-83d4-373bce4dfeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:50.134933 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc8a55-e9f7-46bd-83d4-373bce4dfeba admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.135207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc8a55-e9f7-46bd-83d4-373bce4dfeba admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.135470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc8a55-e9f7-46bd-83d4-373bce4dfeba admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.136101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc8a55-e9f7-46bd-83d4-373bce4dfeba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:50.137713 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc8a55-e9f7-46bd-83d4-373bce4dfeba admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:50.150052 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1243/2480] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:50.154039 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82cba1-88b4-43ef-871e-c13ccf42771f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:50.165250 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82cba1-88b4-43ef-871e-c13ccf42771f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:50.169748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf021275-b2f2-4500-ab9e-c68475ed65b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:50.170510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf021275-b2f2-4500-ab9e-c68475ed65b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.170789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf021275-b2f2-4500-ab9e-c68475ed65b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.171056 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf021275-b2f2-4500-ab9e-c68475ed65b0 admin admin] PATH_INFO: `/v3/projects/ce66141aa9874e948055058eed1450d3/users/873c768c5f91446b9eff07d76995bf5f/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.185343 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf021275-b2f2-4500-ab9e-c68475ed65b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce66141aa9874e948055058eed1450d3, user_id=873c768c5f91446b9eff07d76995bf5f, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:50.188321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf021275-b2f2-4500-ab9e-c68475ed65b0 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:50.190293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:50.203094 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1238/2481] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:50 2022] PUT /identity/v3/projects/ce66141aa9874e948055058eed1450d3/users/873c768c5f91446b9eff07d76995bf5f/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:50.214296 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966d722-38e1-4b94-80a6-b992e0734a6e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.214605 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966d722-38e1-4b94-80a6-b992e0734a6e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.214875 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966d722-38e1-4b94-80a6-b992e0734a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.242861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-6966d722-38e1-4b94-80a6-b992e0734a6e None None] MFA Rules not processed for user `873c768c5f91446b9eff07d76995bf5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:50.244046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-6966d722-38e1-4b94-80a6-b992e0734a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:50.309422 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1244/2482] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:50.314447 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf021275-b2f2-4500-ab9e-c68475ed65b0 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:50.374590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf021275-b2f2-4500-ab9e-c68475ed65b0 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:50.380152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3726854-a9bb-4d35-a49d-e557bb1de535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:50.380778 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3726854-a9bb-4d35-a49d-e557bb1de535 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.381186 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3726854-a9bb-4d35-a49d-e557bb1de535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.381479 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3726854-a9bb-4d35-a49d-e557bb1de535 admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.382128 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3726854-a9bb-4d35-a49d-e557bb1de535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:50.384047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3726854-a9bb-4d35-a49d-e557bb1de535 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:50.411337 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1239/2483] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:53:50 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:53:50.415519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966d722-38e1-4b94-80a6-b992e0734a6e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:50.426493 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966d722-38e1-4b94-80a6-b992e0734a6e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:50.483488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217680b9-b8bd-449f-968f-1652cbbcca6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:50.484026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217680b9-b8bd-449f-968f-1652cbbcca6e admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.484305 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217680b9-b8bd-449f-968f-1652cbbcca6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.484578 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217680b9-b8bd-449f-968f-1652cbbcca6e admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.485363 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217680b9-b8bd-449f-968f-1652cbbcca6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:50.487307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217680b9-b8bd-449f-968f-1652cbbcca6e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:50.508948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1245/2484] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:53:50 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:53:50.512875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3726854-a9bb-4d35-a49d-e557bb1de535 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:50.523619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3726854-a9bb-4d35-a49d-e557bb1de535 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:50.528152 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cc129-a73c-48d1-84e8-6eb6679fec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:50.528733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cc129-a73c-48d1-84e8-6eb6679fec38 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.529012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cc129-a73c-48d1-84e8-6eb6679fec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.529277 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cc129-a73c-48d1-84e8-6eb6679fec38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.529902 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cc129-a73c-48d1-84e8-6eb6679fec38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:50.531768 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cc129-a73c-48d1-84e8-6eb6679fec38 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:50.542130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1240/2485] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:50.546177 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217680b9-b8bd-449f-968f-1652cbbcca6e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:50.555878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217680b9-b8bd-449f-968f-1652cbbcca6e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:50.560410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e2e14-c6dd-4a21-a193-b983a00f6c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:50.561041 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e2e14-c6dd-4a21-a193-b983a00f6c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.561316 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e2e14-c6dd-4a21-a193-b983a00f6c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.561584 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e2e14-c6dd-4a21-a193-b983a00f6c0e admin admin] PATH_INFO: `/v3/projects/55fa3fe1458f48158f2a0f562b4785bd/users/7133f9d7e3c844d68be30577a7faf52c/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.576518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e2e14-c6dd-4a21-a193-b983a00f6c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55fa3fe1458f48158f2a0f562b4785bd, user_id=7133f9d7e3c844d68be30577a7faf52c, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:50.580091 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e2e14-c6dd-4a21-a193-b983a00f6c0e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:50.592715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1246/2486] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:50 2022] PUT /identity/v3/projects/55fa3fe1458f48158f2a0f562b4785bd/users/7133f9d7e3c844d68be30577a7faf52c/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:50.596736 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cc129-a73c-48d1-84e8-6eb6679fec38 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:50.606558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cc129-a73c-48d1-84e8-6eb6679fec38 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:50.663044 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727cd23-55c6-4686-b4d6-c86a60b5a973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:50.663641 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727cd23-55c6-4686-b4d6-c86a60b5a973 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.663916 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727cd23-55c6-4686-b4d6-c86a60b5a973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.664183 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727cd23-55c6-4686-b4d6-c86a60b5a973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.664816 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727cd23-55c6-4686-b4d6-c86a60b5a973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:50.666416 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727cd23-55c6-4686-b4d6-c86a60b5a973 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:50.675966 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1241/2487] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:53:50.680306 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e2e14-c6dd-4a21-a193-b983a00f6c0e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:50.691391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e2e14-c6dd-4a21-a193-b983a00f6c0e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:50.696021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377ca95-3fc8-441d-bd75-741ea6f752ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:50.696662 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377ca95-3fc8-441d-bd75-741ea6f752ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.696937 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377ca95-3fc8-441d-bd75-741ea6f752ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.697207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377ca95-3fc8-441d-bd75-741ea6f752ee admin admin] PATH_INFO: `/v3/projects/55fa3fe1458f48158f2a0f562b4785bd/users/7133f9d7e3c844d68be30577a7faf52c/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.699133 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377ca95-3fc8-441d-bd75-741ea6f752ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=55fa3fe1458f48158f2a0f562b4785bd, user_id=7133f9d7e3c844d68be30577a7faf52c, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:50.701722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377ca95-3fc8-441d-bd75-741ea6f752ee admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:50.712473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1247/2488] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:53:50 2022] PUT /identity/v3/projects/55fa3fe1458f48158f2a0f562b4785bd/users/7133f9d7e3c844d68be30577a7faf52c/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:53:50.724103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4183d-c3c9-4ec5-921c-bea4a1d09799 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.724575 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4183d-c3c9-4ec5-921c-bea4a1d09799 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.724699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4183d-c3c9-4ec5-921c-bea4a1d09799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.753125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-79b4183d-c3c9-4ec5-921c-bea4a1d09799 None None] MFA Rules not processed for user `7133f9d7e3c844d68be30577a7faf52c`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:53:50.754286 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-79b4183d-c3c9-4ec5-921c-bea4a1d09799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:50.802818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:50.822396 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1242/2489] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:53:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:53:50.833101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377ca95-3fc8-441d-bd75-741ea6f752ee admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:50.834914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-9377ca95-3fc8-441d-bd75-741ea6f752ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:53:50.857391 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:53:50.894651 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377ca95-3fc8-441d-bd75-741ea6f752ee admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:50.898269 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6c1c1-a1cd-49f7-88fc-853111c62d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:50.898865 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6c1c1-a1cd-49f7-88fc-853111c62d68 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:50.899142 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6c1c1-a1cd-49f7-88fc-853111c62d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:50.899410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6c1c1-a1cd-49f7-88fc-853111c62d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:50.954520 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6c1c1-a1cd-49f7-88fc-853111c62d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:50.955932 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6c1c1-a1cd-49f7-88fc-853111c62d68 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:50.964569 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1248/2490] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:53:58.696981 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4183d-c3c9-4ec5-921c-bea4a1d09799 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:53:58.706587 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4183d-c3c9-4ec5-921c-bea4a1d09799 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:53:58.710058 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e751cf8-12ec-4c11-8144-42e39799988a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:53:58.710680 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e751cf8-12ec-4c11-8144-42e39799988a service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:53:58.710957 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e751cf8-12ec-4c11-8144-42e39799988a service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:53:58.711223 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e751cf8-12ec-4c11-8144-42e39799988a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:53:58.720527 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e751cf8-12ec-4c11-8144-42e39799988a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:53:58.721873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e751cf8-12ec-4c11-8144-42e39799988a service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:53:58.731101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1243/2491] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:53:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:54:00.960705 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=39 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:06.209125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6c1c1-a1cd-49f7-88fc-853111c62d68 service cinder] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:06.210913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-07b6c1c1-a1cd-49f7-88fc-853111c62d68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:06.266459 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6c1c1-a1cd-49f7-88fc-853111c62d68 service cinder] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:06.270039 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f39bca-1732-4a57-8196-e9d5705c9445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:06.270667 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39bca-1732-4a57-8196-e9d5705c9445 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:06.270948 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39bca-1732-4a57-8196-e9d5705c9445 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:06.271221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39bca-1732-4a57-8196-e9d5705c9445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:06.273140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-43f39bca-1732-4a57-8196-e9d5705c9445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:06.282846 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f39bca-1732-4a57-8196-e9d5705c9445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:06.284911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f39bca-1732-4a57-8196-e9d5705c9445 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:06.292720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1249/2492] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:54:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:54:07.465619 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e751cf8-12ec-4c11-8144-42e39799988a service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:07.467188 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3e751cf8-12ec-4c11-8144-42e39799988a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:07.544684 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e751cf8-12ec-4c11-8144-42e39799988a service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:07.550631 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b1ae8-bf37-4773-882d-62b97c9873ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:07.551194 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b1ae8-bf37-4773-882d-62b97c9873ea service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:07.551475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b1ae8-bf37-4773-882d-62b97c9873ea service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:07.551746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b1ae8-bf37-4773-882d-62b97c9873ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:07.553643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-c60b1ae8-bf37-4773-882d-62b97c9873ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:07.568979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b1ae8-bf37-4773-882d-62b97c9873ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:07.570965 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b1ae8-bf37-4773-882d-62b97c9873ea service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:07.583291 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1244/2493] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:54:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:54:13.554626 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22000ad-fe1e-4900-9104-1b63de2aa182 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:13.555115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22000ad-fe1e-4900-9104-1b63de2aa182 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:13.555218 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22000ad-fe1e-4900-9104-1b63de2aa182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:13.574475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-c22000ad-fe1e-4900-9104-1b63de2aa182 None None] MFA Rules not processed for user `568d031158744101a6bb339e51149ed2`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:13.575592 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c22000ad-fe1e-4900-9104-1b63de2aa182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:13.630588 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1250/2494] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:17.577730 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=45 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:23.623214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=51 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:29.227443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b1ae8-bf37-4773-882d-62b97c9873ea service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:29.293822 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b1ae8-bf37-4773-882d-62b97c9873ea service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:29.299049 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:29.299709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:29.299969 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:29.300246 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] PATH_INFO: `/v3/users/873c768c5f91446b9eff07d76995bf5f` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:29.301372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=873c768c5f91446b9eff07d76995bf5f)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:29.303321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:29.365330 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '873c768c5f91446b9eff07d76995bf5f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:29.365997 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '873c768c5f91446b9eff07d76995bf5f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:29.366657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '873c768c5f91446b9eff07d76995bf5f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:29.373776 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '873c768c5f91446b9eff07d76995bf5f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:29.386100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '873c768c5f91446b9eff07d76995bf5f'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:29.399424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1245/2495] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:54:29 2022] DELETE /identity/v3/users/873c768c5f91446b9eff07d76995bf5f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:29.403988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566630-7a1d-4e05-8f10-bf204869fe89 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:29.404484 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566630-7a1d-4e05-8f10-bf204869fe89 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:29.404752 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566630-7a1d-4e05-8f10-bf204869fe89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:29.425945 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-77566630-7a1d-4e05-8f10-bf204869fe89 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:29.427221 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-77566630-7a1d-4e05-8f10-bf204869fe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:29.491489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1251/2496] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:29.501450 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:29.503113 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:29.563539 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7bb6a-6759-436c-8927-fad8e09ef5cc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:29.568325 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c649c-9a0a-4e8d-8352-e811fec19631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:29.568943 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c649c-9a0a-4e8d-8352-e811fec19631 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:29.569224 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c649c-9a0a-4e8d-8352-e811fec19631 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:29.569645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c649c-9a0a-4e8d-8352-e811fec19631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:29.579091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c649c-9a0a-4e8d-8352-e811fec19631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:29.580581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c649c-9a0a-4e8d-8352-e811fec19631 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:29.590442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1246/2497] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:54:29.858972 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77566630-7a1d-4e05-8f10-bf204869fe89 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:29.860423 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-77566630-7a1d-4e05-8f10-bf204869fe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:29.875741 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77566630-7a1d-4e05-8f10-bf204869fe89 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:29.881181 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:29.882454 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:29.882921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:29.883204 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] PATH_INFO: `/v3/projects/ce66141aa9874e948055058eed1450d3` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:29.884488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce66141aa9874e948055058eed1450d3)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:29.886426 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:29.934930 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce66141aa9874e948055058eed1450d3', 'request_id': 'req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:29.940462 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce66141aa9874e948055058eed1450d3', 'request_id': 'req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:29.947756 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce66141aa9874e948055058eed1450d3', 'request_id': 'req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:29.959128 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] The token cache is being invalidate because project ce66141aa9874e948055058eed1450d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:54:29.959410 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:29.960019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:29.961469 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1252/2498] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:54:29 2022] DELETE /identity/v3/projects/ce66141aa9874e948055058eed1450d3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:29.965388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c649c-9a0a-4e8d-8352-e811fec19631 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:29.975389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c649c-9a0a-4e8d-8352-e811fec19631 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:29.993801 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:30.036516 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96848754-0ca6-4057-aa01-d3eeded02e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:30.037144 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96848754-0ca6-4057-aa01-d3eeded02e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:30.037419 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96848754-0ca6-4057-aa01-d3eeded02e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:30.037687 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96848754-0ca6-4057-aa01-d3eeded02e01 admin admin] PATH_INFO: `/v3/users/7133f9d7e3c844d68be30577a7faf52c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:30.038815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96848754-0ca6-4057-aa01-d3eeded02e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7133f9d7e3c844d68be30577a7faf52c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:30.040725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96848754-0ca6-4057-aa01-d3eeded02e01 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:30.089643 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-96848754-0ca6-4057-aa01-d3eeded02e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7133f9d7e3c844d68be30577a7faf52c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:30.090317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-96848754-0ca6-4057-aa01-d3eeded02e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7133f9d7e3c844d68be30577a7faf52c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:30.090912 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-96848754-0ca6-4057-aa01-d3eeded02e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7133f9d7e3c844d68be30577a7faf52c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:30.098118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-96848754-0ca6-4057-aa01-d3eeded02e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7133f9d7e3c844d68be30577a7faf52c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:30.110320 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-96848754-0ca6-4057-aa01-d3eeded02e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7133f9d7e3c844d68be30577a7faf52c'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:30.123454 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1247/2499] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:54:29 2022] DELETE /identity/v3/users/7133f9d7e3c844d68be30577a7faf52c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:30.409929 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:30.411438 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:30.425278 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbbebd-9a0b-410a-b102-e71d4bdb6d5a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:30.481644 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:30.482311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:30.482591 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:30.482862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] PATH_INFO: `/v3/projects/55fa3fe1458f48158f2a0f562b4785bd` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:30.483956 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55fa3fe1458f48158f2a0f562b4785bd)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:30.486038 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:30.516307 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:30.535514 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55fa3fe1458f48158f2a0f562b4785bd', 'request_id': 'req-f3693c06-77af-466c-88a8-6027ce652686'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:30.540878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55fa3fe1458f48158f2a0f562b4785bd', 'request_id': 'req-f3693c06-77af-466c-88a8-6027ce652686'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:30.547733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55fa3fe1458f48158f2a0f562b4785bd', 'request_id': 'req-f3693c06-77af-466c-88a8-6027ce652686'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:30.558913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] The token cache is being invalidate because project 55fa3fe1458f48158f2a0f562b4785bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:54:30.559197 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:30.559824 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-f3693c06-77af-466c-88a8-6027ce652686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:30.561297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1253/2500] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:54:30 2022] DELETE /identity/v3/projects/55fa3fe1458f48158f2a0f562b4785bd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:30.566316 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c66b6c-b2b2-4f72-80d6-f9f5c816839f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:30.566818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c66b6c-b2b2-4f72-80d6-f9f5c816839f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:30.566945 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c66b6c-b2b2-4f72-80d6-f9f5c816839f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:30.586942 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-b4c66b6c-b2b2-4f72-80d6-f9f5c816839f None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:30.588127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-b4c66b6c-b2b2-4f72-80d6-f9f5c816839f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:30.646229 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1248/2501] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:30.654205 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55437dc-a569-44b2-9549-19e16d03a703 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:30.654555 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55437dc-a569-44b2-9549-19e16d03a703 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:30.656403 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55437dc-a569-44b2-9549-19e16d03a703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:30.676341 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-f55437dc-a569-44b2-9549-19e16d03a703 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:30.677549 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-f55437dc-a569-44b2-9549-19e16d03a703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:30.685600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1254/2502] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:30.689886 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c66b6c-b2b2-4f72-80d6-f9f5c816839f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:30.700216 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c66b6c-b2b2-4f72-80d6-f9f5c816839f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:30.705482 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309f9f4-d560-439f-b3f2-d8743bc6c7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:30.706076 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309f9f4-d560-439f-b3f2-d8743bc6c7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:30.706388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309f9f4-d560-439f-b3f2-d8743bc6c7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:30.706663 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309f9f4-d560-439f-b3f2-d8743bc6c7c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:30.707312 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309f9f4-d560-439f-b3f2-d8743bc6c7c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:54:30.707613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309f9f4-d560-439f-b3f2-d8743bc6c7c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:30.709197 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309f9f4-d560-439f-b3f2-d8743bc6c7c3 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:30.721880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1249/2503] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:54:30.726294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55437dc-a569-44b2-9549-19e16d03a703 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:30.735860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55437dc-a569-44b2-9549-19e16d03a703 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:30.740607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777c3ab-364a-469c-95d0-3b42aa283f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:30.741207 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777c3ab-364a-469c-95d0-3b42aa283f07 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:30.741491 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777c3ab-364a-469c-95d0-3b42aa283f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:30.741761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777c3ab-364a-469c-95d0-3b42aa283f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:30.742427 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777c3ab-364a-469c-95d0-3b42aa283f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:30.744352 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777c3ab-364a-469c-95d0-3b42aa283f07 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:30.775253 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1255/2504] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:54:30 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:54:30.776440 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309f9f4-d560-439f-b3f2-d8743bc6c7c3 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:30.787657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309f9f4-d560-439f-b3f2-d8743bc6c7c3 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:30.815021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:30.848162 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3fab6-ab56-4cb0-ad30-c8bd61bc354e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:30.848760 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3fab6-ab56-4cb0-ad30-c8bd61bc354e admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:30.849037 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3fab6-ab56-4cb0-ad30-c8bd61bc354e admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:30.849305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3fab6-ab56-4cb0-ad30-c8bd61bc354e admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:30.849921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3fab6-ab56-4cb0-ad30-c8bd61bc354e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:30.851850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3fab6-ab56-4cb0-ad30-c8bd61bc354e admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:30.872646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1250/2505] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:54:30 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:54:30.877292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777c3ab-364a-469c-95d0-3b42aa283f07 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:30.887600 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777c3ab-364a-469c-95d0-3b42aa283f07 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:30.892860 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089193a-31f0-4d22-acf8-fe37330d1153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:30.893468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089193a-31f0-4d22-acf8-fe37330d1153 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:30.893749 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089193a-31f0-4d22-acf8-fe37330d1153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:30.894033 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089193a-31f0-4d22-acf8-fe37330d1153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:30.894695 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089193a-31f0-4d22-acf8-fe37330d1153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:30.896276 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089193a-31f0-4d22-acf8-fe37330d1153 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:30.906572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1256/2506] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:54:30.910706 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3fab6-ab56-4cb0-ad30-c8bd61bc354e admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:30.920973 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3fab6-ab56-4cb0-ad30-c8bd61bc354e admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:30.925580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d949a-c644-432e-8743-e0245cb24c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:30.926187 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d949a-c644-432e-8743-e0245cb24c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:30.926464 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d949a-c644-432e-8743-e0245cb24c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:30.926733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d949a-c644-432e-8743-e0245cb24c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:30.927364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d949a-c644-432e-8743-e0245cb24c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:30.928923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d949a-c644-432e-8743-e0245cb24c39 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:30.940040 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1251/2507] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:54:30.944011 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089193a-31f0-4d22-acf8-fe37330d1153 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:30.954047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089193a-31f0-4d22-acf8-fe37330d1153 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:30.959330 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23eb29-a6b8-4466-b29f-5eef50c97a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:30.959978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23eb29-a6b8-4466-b29f-5eef50c97a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:30.960259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23eb29-a6b8-4466-b29f-5eef50c97a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:30.960532 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23eb29-a6b8-4466-b29f-5eef50c97a11 admin admin] PATH_INFO: `/v3/projects/3f93cd1ad2f045d5a4963cf273444cdb/users/c9783e9b6c374480a0f0bff3dac8d062/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:30.976590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23eb29-a6b8-4466-b29f-5eef50c97a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f93cd1ad2f045d5a4963cf273444cdb, user_id=c9783e9b6c374480a0f0bff3dac8d062, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:30.979412 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23eb29-a6b8-4466-b29f-5eef50c97a11 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:30.994564 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1257/2508] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:54:30 2022] PUT /identity/v3/projects/3f93cd1ad2f045d5a4963cf273444cdb/users/c9783e9b6c374480a0f0bff3dac8d062/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:31.006780 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b4cb7-61f4-4925-8df4-58d36a88ddf2 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:31.007103 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b4cb7-61f4-4925-8df4-58d36a88ddf2 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:31.007436 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b4cb7-61f4-4925-8df4-58d36a88ddf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:31.037317 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1b3b4cb7-61f4-4925-8df4-58d36a88ddf2 None None] MFA Rules not processed for user `c9783e9b6c374480a0f0bff3dac8d062`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:31.038985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1b3b4cb7-61f4-4925-8df4-58d36a88ddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:31.104158 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1252/2509] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:31.108587 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23eb29-a6b8-4466-b29f-5eef50c97a11 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:31.170277 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23eb29-a6b8-4466-b29f-5eef50c97a11 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:31.175757 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff87898-2a70-4d6b-8e30-57f29df60605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:31.176356 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87898-2a70-4d6b-8e30-57f29df60605 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:31.176635 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87898-2a70-4d6b-8e30-57f29df60605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:31.176903 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87898-2a70-4d6b-8e30-57f29df60605 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:31.177522 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff87898-2a70-4d6b-8e30-57f29df60605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:31.179432 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff87898-2a70-4d6b-8e30-57f29df60605 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:31.206633 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1258/2510] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:54:31 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:54:31.210793 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b4cb7-61f4-4925-8df4-58d36a88ddf2 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:31.222659 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b4cb7-61f4-4925-8df4-58d36a88ddf2 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:31.278107 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc84b1-26e6-4394-9f29-8a5eb285bcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:31.278742 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc84b1-26e6-4394-9f29-8a5eb285bcab admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:31.279023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc84b1-26e6-4394-9f29-8a5eb285bcab admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:31.279297 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc84b1-26e6-4394-9f29-8a5eb285bcab admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:31.279924 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc84b1-26e6-4394-9f29-8a5eb285bcab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:31.282093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc84b1-26e6-4394-9f29-8a5eb285bcab admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:31.303598 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1253/2511] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:54:31 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:54:31.307577 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff87898-2a70-4d6b-8e30-57f29df60605 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:31.318627 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff87898-2a70-4d6b-8e30-57f29df60605 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:31.331418 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bedee-a0b5-4473-90c7-c72e6cb2f27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:31.331859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bedee-a0b5-4473-90c7-c72e6cb2f27d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:31.332153 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bedee-a0b5-4473-90c7-c72e6cb2f27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:31.332423 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bedee-a0b5-4473-90c7-c72e6cb2f27d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:31.333072 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bedee-a0b5-4473-90c7-c72e6cb2f27d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:31.334711 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bedee-a0b5-4473-90c7-c72e6cb2f27d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:31.346671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1259/2512] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:54:31.350657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc84b1-26e6-4394-9f29-8a5eb285bcab admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:31.360839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc84b1-26e6-4394-9f29-8a5eb285bcab admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:31.365845 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b6cf9-fe09-42c1-82c8-29345efd34c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:31.366537 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b6cf9-fe09-42c1-82c8-29345efd34c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:31.366818 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b6cf9-fe09-42c1-82c8-29345efd34c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:31.367090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b6cf9-fe09-42c1-82c8-29345efd34c1 admin admin] PATH_INFO: `/v3/projects/46f8f5af9f0448f79427be7f98b609b5/users/6a6a1f78fa7e4108b5c8485229d3525e/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:31.382755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b6cf9-fe09-42c1-82c8-29345efd34c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f8f5af9f0448f79427be7f98b609b5, user_id=6a6a1f78fa7e4108b5c8485229d3525e, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:31.385699 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b6cf9-fe09-42c1-82c8-29345efd34c1 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:31.401930 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1254/2513] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:54:31 2022] PUT /identity/v3/projects/46f8f5af9f0448f79427be7f98b609b5/users/6a6a1f78fa7e4108b5c8485229d3525e/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:54:31.406096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bedee-a0b5-4473-90c7-c72e6cb2f27d admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:31.418460 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bedee-a0b5-4473-90c7-c72e6cb2f27d admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:31.438955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:31.479969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d27d58-1fe3-4f2a-9e7e-7b054de086f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:31.480582 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d27d58-1fe3-4f2a-9e7e-7b054de086f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:31.480859 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d27d58-1fe3-4f2a-9e7e-7b054de086f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:31.481130 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d27d58-1fe3-4f2a-9e7e-7b054de086f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:31.481768 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d27d58-1fe3-4f2a-9e7e-7b054de086f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:31.483406 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d27d58-1fe3-4f2a-9e7e-7b054de086f1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:31.493357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1260/2514] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:54:31.497272 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b6cf9-fe09-42c1-82c8-29345efd34c1 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:31.503218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:31.509970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b6cf9-fe09-42c1-82c8-29345efd34c1 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:31.515218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a3a67-196f-49f9-bf7b-8a9efc03a557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:31.515975 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a3a67-196f-49f9-bf7b-8a9efc03a557 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:31.516326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a3a67-196f-49f9-bf7b-8a9efc03a557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:31.516686 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a3a67-196f-49f9-bf7b-8a9efc03a557 admin admin] PATH_INFO: `/v3/projects/46f8f5af9f0448f79427be7f98b609b5/users/6a6a1f78fa7e4108b5c8485229d3525e/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:31.519905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a3a67-196f-49f9-bf7b-8a9efc03a557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f8f5af9f0448f79427be7f98b609b5, user_id=6a6a1f78fa7e4108b5c8485229d3525e, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:31.522361 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a3a67-196f-49f9-bf7b-8a9efc03a557 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:31.535738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1255/2515] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:54:31 2022] PUT /identity/v3/projects/46f8f5af9f0448f79427be7f98b609b5/users/6a6a1f78fa7e4108b5c8485229d3525e/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:31.545902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57ada0-dca9-4f6a-98dc-170f130b147e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:31.546214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57ada0-dca9-4f6a-98dc-170f130b147e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:31.546499 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57ada0-dca9-4f6a-98dc-170f130b147e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:31.575069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-5d57ada0-dca9-4f6a-98dc-170f130b147e None None] MFA Rules not processed for user `6a6a1f78fa7e4108b5c8485229d3525e`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:31.576196 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5d57ada0-dca9-4f6a-98dc-170f130b147e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:31.649607 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1261/2516] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:31.661340 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a3a67-196f-49f9-bf7b-8a9efc03a557 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:31.663092 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-6a0a3a67-196f-49f9-bf7b-8a9efc03a557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:31.679482 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57ada0-dca9-4f6a-98dc-170f130b147e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:31.680869 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5d57ada0-dca9-4f6a-98dc-170f130b147e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:31.723081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a3a67-196f-49f9-bf7b-8a9efc03a557 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:31.726999 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9012af-c848-43fb-973d-773ab9eea1d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:31.727601 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9012af-c848-43fb-973d-773ab9eea1d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:31.727875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9012af-c848-43fb-973d-773ab9eea1d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:31.728147 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9012af-c848-43fb-973d-773ab9eea1d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:31.737046 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9012af-c848-43fb-973d-773ab9eea1d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:31.738565 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9012af-c848-43fb-973d-773ab9eea1d9 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:31.738704 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57ada0-dca9-4f6a-98dc-170f130b147e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:31.742480 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5418a-20df-4324-a0e8-cb2eed11fc27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:31.743096 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5418a-20df-4324-a0e8-cb2eed11fc27 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:31.743877 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5418a-20df-4324-a0e8-cb2eed11fc27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:31.744154 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5418a-20df-4324-a0e8-cb2eed11fc27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:31.745854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-33f5418a-20df-4324-a0e8-cb2eed11fc27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:31.748086 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1256/2517] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:54:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:54:31.800531 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9012af-c848-43fb-973d-773ab9eea1d9 service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:31.802192 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ba9012af-c848-43fb-973d-773ab9eea1d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:31.811892 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:31.814208 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9012af-c848-43fb-973d-773ab9eea1d9 service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:31.815042 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5418a-20df-4324-a0e8-cb2eed11fc27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:31.816714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5418a-20df-4324-a0e8-cb2eed11fc27 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:31.818023 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fd9f4-f44b-4d3b-b8e5-a4d114871e1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:31.818648 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fd9f4-f44b-4d3b-b8e5-a4d114871e1b service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:31.818927 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fd9f4-f44b-4d3b-b8e5-a4d114871e1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:31.819196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fd9f4-f44b-4d3b-b8e5-a4d114871e1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:31.826825 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1262/2518] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:54:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:54:31.884398 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fd9f4-f44b-4d3b-b8e5-a4d114871e1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:31.885907 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fd9f4-f44b-4d3b-b8e5-a4d114871e1b service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:31.895894 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1257/2519] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:54:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:54:33.747721 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5418a-20df-4324-a0e8-cb2eed11fc27 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:33.750673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-33f5418a-20df-4324-a0e8-cb2eed11fc27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:33.856256 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5418a-20df-4324-a0e8-cb2eed11fc27 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:33.865262 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e7340-092f-4489-9b1f-583d174835bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:33.866035 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e7340-092f-4489-9b1f-583d174835bc service nova] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:33.866428 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e7340-092f-4489-9b1f-583d174835bc service nova] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:33.866603 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e7340-092f-4489-9b1f-583d174835bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:33.868671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-e17e7340-092f-4489-9b1f-583d174835bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:33.884146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e7340-092f-4489-9b1f-583d174835bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:33.885772 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e7340-092f-4489-9b1f-583d174835bc service nova] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:33.899314 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1263/2520] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:54:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:54:35.482091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fd9f4-f44b-4d3b-b8e5-a4d114871e1b service cinder] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:35.544444 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fd9f4-f44b-4d3b-b8e5-a4d114871e1b service cinder] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:35.549585 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:35.550239 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:35.550521 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:35.550791 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] PATH_INFO: `/v3/users/c9783e9b6c374480a0f0bff3dac8d062` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:35.552024 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9783e9b6c374480a0f0bff3dac8d062)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:35.554082 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:35.594135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:35.613185 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9783e9b6c374480a0f0bff3dac8d062'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:35.617041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9783e9b6c374480a0f0bff3dac8d062'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:35.617922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9783e9b6c374480a0f0bff3dac8d062'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:35.626227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9783e9b6c374480a0f0bff3dac8d062'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:35.643733 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9783e9b6c374480a0f0bff3dac8d062'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:35.659796 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1258/2521] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:54:35 2022] DELETE /identity/v3/users/c9783e9b6c374480a0f0bff3dac8d062 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:35.664373 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381b3ff-bc69-40ce-8273-bc6201d4e714 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:35.664862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381b3ff-bc69-40ce-8273-bc6201d4e714 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:35.665618 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381b3ff-bc69-40ce-8273-bc6201d4e714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:35.692636 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-7381b3ff-bc69-40ce-8273-bc6201d4e714 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:35.694814 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7381b3ff-bc69-40ce-8273-bc6201d4e714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:35.760371 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1264/2522] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:54:35.770865 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:35.772294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:35.830823 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061e591-768b-40fc-881a-9c9a0e4411ae admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:35.834664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d972d70-5425-43d0-b0ed-8f6be8c6dc0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:35.835294 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d972d70-5425-43d0-b0ed-8f6be8c6dc0b service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:35.835574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d972d70-5425-43d0-b0ed-8f6be8c6dc0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:35.835840 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d972d70-5425-43d0-b0ed-8f6be8c6dc0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:35.846759 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d972d70-5425-43d0-b0ed-8f6be8c6dc0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:35.848214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d972d70-5425-43d0-b0ed-8f6be8c6dc0b service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:35.857627 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1259/2523] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:54:36.122058 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381b3ff-bc69-40ce-8273-bc6201d4e714 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:36.123537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7381b3ff-bc69-40ce-8273-bc6201d4e714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:36.136040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381b3ff-bc69-40ce-8273-bc6201d4e714 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:36.140759 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:36.141466 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:36.141962 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:36.143167 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] PATH_INFO: `/v3/projects/3f93cd1ad2f045d5a4963cf273444cdb` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:36.143851 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f93cd1ad2f045d5a4963cf273444cdb)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:36.145776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:36.192754 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f93cd1ad2f045d5a4963cf273444cdb', 'request_id': 'req-c89d0930-25fa-4594-be7b-771451be5379'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.198340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f93cd1ad2f045d5a4963cf273444cdb', 'request_id': 'req-c89d0930-25fa-4594-be7b-771451be5379'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.206339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f93cd1ad2f045d5a4963cf273444cdb', 'request_id': 'req-c89d0930-25fa-4594-be7b-771451be5379'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.217679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] The token cache is being invalidate because project 3f93cd1ad2f045d5a4963cf273444cdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:54:36.217966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.218640 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.220158 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1265/2524] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:54:36 2022] DELETE /identity/v3/projects/3f93cd1ad2f045d5a4963cf273444cdb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:36.224099 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d972d70-5425-43d0-b0ed-8f6be8c6dc0b service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:36.233668 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d972d70-5425-43d0-b0ed-8f6be8c6dc0b service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:36.289454 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ac6eb-b751-4def-aa88-30e41250faa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:36.290069 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac6eb-b751-4def-aa88-30e41250faa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:36.290378 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac6eb-b751-4def-aa88-30e41250faa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:36.290652 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac6eb-b751-4def-aa88-30e41250faa2 admin admin] PATH_INFO: `/v3/users/6a6a1f78fa7e4108b5c8485229d3525e` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:36.291761 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ac6eb-b751-4def-aa88-30e41250faa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a6a1f78fa7e4108b5c8485229d3525e)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:36.293693 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ac6eb-b751-4def-aa88-30e41250faa2 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:36.339025 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-951ac6eb-b751-4def-aa88-30e41250faa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a6a1f78fa7e4108b5c8485229d3525e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.339670 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-951ac6eb-b751-4def-aa88-30e41250faa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a6a1f78fa7e4108b5c8485229d3525e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.340279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-951ac6eb-b751-4def-aa88-30e41250faa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a6a1f78fa7e4108b5c8485229d3525e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.345985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:36.348276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-951ac6eb-b751-4def-aa88-30e41250faa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a6a1f78fa7e4108b5c8485229d3525e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.362112 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-951ac6eb-b751-4def-aa88-30e41250faa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a6a1f78fa7e4108b5c8485229d3525e'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.375847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1260/2525] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:54:36 2022] DELETE /identity/v3/users/6a6a1f78fa7e4108b5c8485229d3525e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:36.650119 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:36.651549 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:36.662054 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:36.666680 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d0930-25fa-4594-be7b-771451be5379 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:36.729283 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:36.729913 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:36.730211 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:36.730483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] PATH_INFO: `/v3/projects/46f8f5af9f0448f79427be7f98b609b5` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:36.731582 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f8f5af9f0448f79427be7f98b609b5)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:36.733497 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:36.783504 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f8f5af9f0448f79427be7f98b609b5', 'request_id': 'req-af52bc52-f118-4ff3-ac54-7ee82e4978e3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.789047 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f8f5af9f0448f79427be7f98b609b5', 'request_id': 'req-af52bc52-f118-4ff3-ac54-7ee82e4978e3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.797347 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f8f5af9f0448f79427be7f98b609b5', 'request_id': 'req-af52bc52-f118-4ff3-ac54-7ee82e4978e3'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.809966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] The token cache is being invalidate because project 46f8f5af9f0448f79427be7f98b609b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:54:36.810287 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.810931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-af52bc52-f118-4ff3-ac54-7ee82e4978e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:36.812402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1266/2526] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:54:36 2022] DELETE /identity/v3/projects/46f8f5af9f0448f79427be7f98b609b5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:36.817574 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bed9c1-b2fe-4eb8-ba76-b06e6d8a4db1 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:36.818081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bed9c1-b2fe-4eb8-ba76-b06e6d8a4db1 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:36.818225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bed9c1-b2fe-4eb8-ba76-b06e6d8a4db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:36.837339 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-55bed9c1-b2fe-4eb8-ba76-b06e6d8a4db1 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:36.838566 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-55bed9c1-b2fe-4eb8-ba76-b06e6d8a4db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:36.899264 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1261/2527] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:36.907338 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe25fb-7794-4800-a7c4-baaefeb5760d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:36.908254 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe25fb-7794-4800-a7c4-baaefeb5760d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:36.908601 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe25fb-7794-4800-a7c4-baaefeb5760d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:36.928904 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-d6fe25fb-7794-4800-a7c4-baaefeb5760d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:36.930086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d6fe25fb-7794-4800-a7c4-baaefeb5760d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:36.937879 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1267/2528] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:36.942399 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bed9c1-b2fe-4eb8-ba76-b06e6d8a4db1 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:36.952032 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bed9c1-b2fe-4eb8-ba76-b06e6d8a4db1 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:36.956906 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211b55-70e6-4655-b80e-5a6065108a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:36.957497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211b55-70e6-4655-b80e-5a6065108a74 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:36.957774 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211b55-70e6-4655-b80e-5a6065108a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:36.958044 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211b55-70e6-4655-b80e-5a6065108a74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:36.958758 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19211b55-70e6-4655-b80e-5a6065108a74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:54:36.959063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19211b55-70e6-4655-b80e-5a6065108a74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:36.960640 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19211b55-70e6-4655-b80e-5a6065108a74 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:36.972452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1262/2529] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:54:36.976640 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe25fb-7794-4800-a7c4-baaefeb5760d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:36.987443 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe25fb-7794-4800-a7c4-baaefeb5760d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:36.992206 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa736a9-85c2-4236-bd96-1cfee43fccde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:36.992794 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa736a9-85c2-4236-bd96-1cfee43fccde admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:36.993067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa736a9-85c2-4236-bd96-1cfee43fccde admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:36.993333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa736a9-85c2-4236-bd96-1cfee43fccde admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:36.993967 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa736a9-85c2-4236-bd96-1cfee43fccde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:36.995942 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa736a9-85c2-4236-bd96-1cfee43fccde admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:37.022778 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1268/2530] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:54:36 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:54:37.026992 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211b55-70e6-4655-b80e-5a6065108a74 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:37.037958 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211b55-70e6-4655-b80e-5a6065108a74 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:37.097091 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d3c12-f7bd-443d-b07c-dc3b0bb3d316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:37.097700 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d3c12-f7bd-443d-b07c-dc3b0bb3d316 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.098000 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d3c12-f7bd-443d-b07c-dc3b0bb3d316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.098296 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d3c12-f7bd-443d-b07c-dc3b0bb3d316 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.098940 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d3c12-f7bd-443d-b07c-dc3b0bb3d316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:37.100867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d3c12-f7bd-443d-b07c-dc3b0bb3d316 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:37.122009 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1263/2531] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:54:37 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:54:37.126251 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa736a9-85c2-4236-bd96-1cfee43fccde admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:37.136667 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa736a9-85c2-4236-bd96-1cfee43fccde admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:37.141387 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2286650-5719-4d2d-be0e-a0f650ee10d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:37.142092 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2286650-5719-4d2d-be0e-a0f650ee10d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.142722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2286650-5719-4d2d-be0e-a0f650ee10d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.143065 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2286650-5719-4d2d-be0e-a0f650ee10d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.143771 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2286650-5719-4d2d-be0e-a0f650ee10d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:37.145480 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2286650-5719-4d2d-be0e-a0f650ee10d4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:37.155881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1269/2532] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:54:37.160362 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d3c12-f7bd-443d-b07c-dc3b0bb3d316 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:37.170412 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d3c12-f7bd-443d-b07c-dc3b0bb3d316 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:37.175480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72700bb-3319-49f4-9d6a-6e988ad2c180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:37.176104 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72700bb-3319-49f4-9d6a-6e988ad2c180 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.176384 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72700bb-3319-49f4-9d6a-6e988ad2c180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.176656 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72700bb-3319-49f4-9d6a-6e988ad2c180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.177323 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72700bb-3319-49f4-9d6a-6e988ad2c180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:37.179012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72700bb-3319-49f4-9d6a-6e988ad2c180 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:37.190460 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1264/2533] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:54:37.194378 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2286650-5719-4d2d-be0e-a0f650ee10d4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:37.216067 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2286650-5719-4d2d-be0e-a0f650ee10d4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:37.223727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ecd05-17aa-4f72-bd72-0b2950c805d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:37.224475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ecd05-17aa-4f72-bd72-0b2950c805d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.224816 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ecd05-17aa-4f72-bd72-0b2950c805d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.225150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ecd05-17aa-4f72-bd72-0b2950c805d5 admin admin] PATH_INFO: `/v3/projects/06f266c989324665b595b55bdf049ed1/users/b657372b461b447ba03ccade7b8e6db8/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.242246 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ecd05-17aa-4f72-bd72-0b2950c805d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f266c989324665b595b55bdf049ed1, user_id=b657372b461b447ba03ccade7b8e6db8, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:37.245004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ecd05-17aa-4f72-bd72-0b2950c805d5 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:37.261284 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1270/2534] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:54:37 2022] PUT /identity/v3/projects/06f266c989324665b595b55bdf049ed1/users/b657372b461b447ba03ccade7b8e6db8/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:37.327616 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df9c6-7e1c-400a-bf6d-98c9e75f707f None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.327922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df9c6-7e1c-400a-bf6d-98c9e75f707f None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.328196 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df9c6-7e1c-400a-bf6d-98c9e75f707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.357880 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-826df9c6-7e1c-400a-bf6d-98c9e75f707f None None] MFA Rules not processed for user `b657372b461b447ba03ccade7b8e6db8`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:37.359125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-826df9c6-7e1c-400a-bf6d-98c9e75f707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:37.387309 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:37.429734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1265/2535] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:37.434157 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ecd05-17aa-4f72-bd72-0b2950c805d5 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:37.469701 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:37.501031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ecd05-17aa-4f72-bd72-0b2950c805d5 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:37.508074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422634b7-4ebd-4376-b901-4eace27d85a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:37.508717 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422634b7-4ebd-4376-b901-4eace27d85a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.509024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422634b7-4ebd-4376-b901-4eace27d85a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.509263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422634b7-4ebd-4376-b901-4eace27d85a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.509992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422634b7-4ebd-4376-b901-4eace27d85a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:37.512204 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422634b7-4ebd-4376-b901-4eace27d85a4 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:37.540555 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1271/2536] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:54:37 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:54:37.544518 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826df9c6-7e1c-400a-bf6d-98c9e75f707f None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:37.555971 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826df9c6-7e1c-400a-bf6d-98c9e75f707f None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:37.611857 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae33a96-fe4b-48bf-8ade-deb5b38cb303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:37.612463 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae33a96-fe4b-48bf-8ade-deb5b38cb303 admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.612744 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae33a96-fe4b-48bf-8ade-deb5b38cb303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.613018 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae33a96-fe4b-48bf-8ade-deb5b38cb303 admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.613657 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae33a96-fe4b-48bf-8ade-deb5b38cb303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:37.615620 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae33a96-fe4b-48bf-8ade-deb5b38cb303 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:37.636035 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1266/2537] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:54:37 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:54:37.640061 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422634b7-4ebd-4376-b901-4eace27d85a4 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:37.650455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422634b7-4ebd-4376-b901-4eace27d85a4 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:37.655116 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515c1b4-b816-4cc8-93be-d138a1b93154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:37.655970 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515c1b4-b816-4cc8-93be-d138a1b93154 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.656250 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515c1b4-b816-4cc8-93be-d138a1b93154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.656519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515c1b4-b816-4cc8-93be-d138a1b93154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.657182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515c1b4-b816-4cc8-93be-d138a1b93154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:37.659377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515c1b4-b816-4cc8-93be-d138a1b93154 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:37.669718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1272/2538] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:54:37.674389 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae33a96-fe4b-48bf-8ade-deb5b38cb303 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:37.684263 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae33a96-fe4b-48bf-8ade-deb5b38cb303 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:37.688838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a2d0c-3952-4b97-aa99-a622c03b1d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:37.689580 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a2d0c-3952-4b97-aa99-a622c03b1d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.689860 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a2d0c-3952-4b97-aa99-a622c03b1d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.690131 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a2d0c-3952-4b97-aa99-a622c03b1d00 admin admin] PATH_INFO: `/v3/projects/1a542fa04d9946fc81152d4b5a70ee7c/users/48fa169df0bf4136baf7861d1281a497/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.704854 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a2d0c-3952-4b97-aa99-a622c03b1d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a542fa04d9946fc81152d4b5a70ee7c, user_id=48fa169df0bf4136baf7861d1281a497, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:37.707674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a2d0c-3952-4b97-aa99-a622c03b1d00 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:37.721348 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1267/2539] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:54:37 2022] PUT /identity/v3/projects/1a542fa04d9946fc81152d4b5a70ee7c/users/48fa169df0bf4136baf7861d1281a497/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:37.725381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515c1b4-b816-4cc8-93be-d138a1b93154 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:37.735350 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515c1b4-b816-4cc8-93be-d138a1b93154 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:37.792530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc4c3c-7e67-4145-9374-e5d2a00ea66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:37.793123 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc4c3c-7e67-4145-9374-e5d2a00ea66d admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.793402 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc4c3c-7e67-4145-9374-e5d2a00ea66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.793667 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc4c3c-7e67-4145-9374-e5d2a00ea66d admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.794325 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc4c3c-7e67-4145-9374-e5d2a00ea66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:37.795896 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc4c3c-7e67-4145-9374-e5d2a00ea66d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:37.806497 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1273/2540] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:54:37.810754 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a2d0c-3952-4b97-aa99-a622c03b1d00 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:37.821675 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a2d0c-3952-4b97-aa99-a622c03b1d00 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:37.826327 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff6364-9808-4953-ba34-436de71edc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:37.826962 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff6364-9808-4953-ba34-436de71edc80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.827243 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff6364-9808-4953-ba34-436de71edc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.827513 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff6364-9808-4953-ba34-436de71edc80 admin admin] PATH_INFO: `/v3/projects/1a542fa04d9946fc81152d4b5a70ee7c/users/48fa169df0bf4136baf7861d1281a497/roles/8b32d66a3cb3483c8ede361ad8637f66` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.829847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff6364-9808-4953-ba34-436de71edc80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a542fa04d9946fc81152d4b5a70ee7c, user_id=48fa169df0bf4136baf7861d1281a497, role_id=8b32d66a3cb3483c8ede361ad8637f66)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:37.832493 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff6364-9808-4953-ba34-436de71edc80 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:37.842504 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1268/2541] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:54:37 2022] PUT /identity/v3/projects/1a542fa04d9946fc81152d4b5a70ee7c/users/48fa169df0bf4136baf7861d1281a497/roles/8b32d66a3cb3483c8ede361ad8637f66 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:37.853083 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3ce1-a000-4b82-b234-3afbc119c8d5 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:37.853380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3ce1-a000-4b82-b234-3afbc119c8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:37.853658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3ce1-a000-4b82-b234-3afbc119c8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:37.882588 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-59fa3ce1-a000-4b82-b234-3afbc119c8d5 None None] MFA Rules not processed for user `48fa169df0bf4136baf7861d1281a497`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:37.883796 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-59fa3ce1-a000-4b82-b234-3afbc119c8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:37.954213 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1274/2542] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:37.966088 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff6364-9808-4953-ba34-436de71edc80 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:37.967779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-5bff6364-9808-4953-ba34-436de71edc80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:38.028936 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff6364-9808-4953-ba34-436de71edc80 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:38.033510 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda835e-2f7d-4861-9158-e7d291782410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:38.034115 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda835e-2f7d-4861-9158-e7d291782410 service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:38.034424 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda835e-2f7d-4861-9158-e7d291782410 service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:38.034694 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda835e-2f7d-4861-9158-e7d291782410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:38.044150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda835e-2f7d-4861-9158-e7d291782410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:38.045608 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda835e-2f7d-4861-9158-e7d291782410 service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:38.055412 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1269/2543] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:54:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:54:38.085773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa3ce1-a000-4b82-b234-3afbc119c8d5 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:38.115490 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:38.149714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa3ce1-a000-4b82-b234-3afbc119c8d5 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:38.154271 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:38.154889 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:38.155162 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:38.155679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] PATH_INFO: `/v3/users/b657372b461b447ba03ccade7b8e6db8` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:38.156786 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b657372b461b447ba03ccade7b8e6db8)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:38.159562 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:38.206082 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b657372b461b447ba03ccade7b8e6db8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:38.206732 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b657372b461b447ba03ccade7b8e6db8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:38.218792 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b657372b461b447ba03ccade7b8e6db8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:38.219537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b657372b461b447ba03ccade7b8e6db8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:38.226845 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b657372b461b447ba03ccade7b8e6db8'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:38.239579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1275/2544] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:54:38 2022] DELETE /identity/v3/users/b657372b461b447ba03ccade7b8e6db8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:54:38.244163 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf26e5-a27b-4457-9820-99628732ddbd None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:38.244480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf26e5-a27b-4457-9820-99628732ddbd None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:38.244751 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf26e5-a27b-4457-9820-99628732ddbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:38.263380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-1caf26e5-a27b-4457-9820-99628732ddbd None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:38.264562 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1caf26e5-a27b-4457-9820-99628732ddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:38.281488 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:38.324919 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1270/2545] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:38.335649 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:38.337143 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:38.395729 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0191a5-f177-4bfc-9da6-a35362aa7087 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:38.399654 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7096ec8-06ff-4f84-ac39-45cb1f88ddac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:38.400254 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096ec8-06ff-4f84-ac39-45cb1f88ddac service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:38.400525 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096ec8-06ff-4f84-ac39-45cb1f88ddac service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:38.400790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096ec8-06ff-4f84-ac39-45cb1f88ddac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:38.410139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7096ec8-06ff-4f84-ac39-45cb1f88ddac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:38.411573 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7096ec8-06ff-4f84-ac39-45cb1f88ddac service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:38.420902 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1276/2546] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:54:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:54:38.926333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf26e5-a27b-4457-9820-99628732ddbd None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:38.927750 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-1caf26e5-a27b-4457-9820-99628732ddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:38.940495 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf26e5-a27b-4457-9820-99628732ddbd None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:38.945118 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:38.945783 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:38.946062 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:38.946364 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] PATH_INFO: `/v3/projects/06f266c989324665b595b55bdf049ed1` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:38.947466 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f266c989324665b595b55bdf049ed1)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:38.949354 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:38.997941 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f266c989324665b595b55bdf049ed1', 'request_id': 'req-e21c149e-7123-4fe4-ba06-b0ee14db02f8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.006130 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f266c989324665b595b55bdf049ed1', 'request_id': 'req-e21c149e-7123-4fe4-ba06-b0ee14db02f8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.012289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f266c989324665b595b55bdf049ed1', 'request_id': 'req-e21c149e-7123-4fe4-ba06-b0ee14db02f8'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.023716 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] The token cache is being invalidate because project 06f266c989324665b595b55bdf049ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:54:39.024014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.024664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.026143 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1271/2547] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:54:38 2022] DELETE /identity/v3/projects/06f266c989324665b595b55bdf049ed1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:39.030093 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7096ec8-06ff-4f84-ac39-45cb1f88ddac service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:39.041380 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7096ec8-06ff-4f84-ac39-45cb1f88ddac service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:39.087046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:39.102229 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f35d22-c0eb-4624-8477-a177d2bf216a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:39.103026 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f35d22-c0eb-4624-8477-a177d2bf216a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:39.103244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f35d22-c0eb-4624-8477-a177d2bf216a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:39.103513 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f35d22-c0eb-4624-8477-a177d2bf216a admin admin] PATH_INFO: `/v3/users/48fa169df0bf4136baf7861d1281a497` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:39.105006 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f35d22-c0eb-4624-8477-a177d2bf216a admin admin] RBAC: Authorizing `identity:delete_user(user_id=48fa169df0bf4136baf7861d1281a497)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:39.106657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f35d22-c0eb-4624-8477-a177d2bf216a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:39.161528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-01f35d22-c0eb-4624-8477-a177d2bf216a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48fa169df0bf4136baf7861d1281a497'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.162264 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-01f35d22-c0eb-4624-8477-a177d2bf216a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48fa169df0bf4136baf7861d1281a497'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.175489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-01f35d22-c0eb-4624-8477-a177d2bf216a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48fa169df0bf4136baf7861d1281a497'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.176101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-01f35d22-c0eb-4624-8477-a177d2bf216a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48fa169df0bf4136baf7861d1281a497'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.183838 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-01f35d22-c0eb-4624-8477-a177d2bf216a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48fa169df0bf4136baf7861d1281a497'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.201775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1277/2548] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:54:39 2022] DELETE /identity/v3/users/48fa169df0bf4136baf7861d1281a497 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:39.526275 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:39.527724 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:39.541725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c149e-7123-4fe4-ba06-b0ee14db02f8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:39.598408 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:39.599038 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:39.599318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:39.599591 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] PATH_INFO: `/v3/projects/1a542fa04d9946fc81152d4b5a70ee7c` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:39.600703 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a542fa04d9946fc81152d4b5a70ee7c)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:39.602639 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:39.646909 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:39.648127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a542fa04d9946fc81152d4b5a70ee7c', 'request_id': 'req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.655654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a542fa04d9946fc81152d4b5a70ee7c', 'request_id': 'req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.662597 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a542fa04d9946fc81152d4b5a70ee7c', 'request_id': 'req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.672635 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] The token cache is being invalidate because project 1a542fa04d9946fc81152d4b5a70ee7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:54:39.672921 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.674072 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3843ef50-1d2a-4b4e-ada5-8746a0f9cf15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:54:39.675569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1272/2549] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:54:39 2022] DELETE /identity/v3/projects/1a542fa04d9946fc81152d4b5a70ee7c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:39.680574 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9d92-225c-42bc-9c99-6021d33bab36 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:39.680862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9d92-225c-42bc-9c99-6021d33bab36 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:39.681132 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9d92-225c-42bc-9c99-6021d33bab36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:39.700421 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-11fa9d92-225c-42bc-9c99-6021d33bab36 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:39.701608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-11fa9d92-225c-42bc-9c99-6021d33bab36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:39.761475 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1278/2550] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:39.769885 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b56ab-3f44-4f2c-a814-e7f45db4fdaa None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:39.770261 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b56ab-3f44-4f2c-a814-e7f45db4fdaa None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:39.770487 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b56ab-3f44-4f2c-a814-e7f45db4fdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:39.790847 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-f88b56ab-3f44-4f2c-a814-e7f45db4fdaa None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:39.792033 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-f88b56ab-3f44-4f2c-a814-e7f45db4fdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:39.799983 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1273/2551] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:39.804215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa9d92-225c-42bc-9c99-6021d33bab36 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:39.814200 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa9d92-225c-42bc-9c99-6021d33bab36 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:39.818855 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578c36c-c562-4121-89a6-fc025835c86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:39.819447 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578c36c-c562-4121-89a6-fc025835c86c admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:39.819722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578c36c-c562-4121-89a6-fc025835c86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:39.819988 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578c36c-c562-4121-89a6-fc025835c86c admin admin] PATH_INFO: `/v3/domains` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:39.820625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578c36c-c562-4121-89a6-fc025835c86c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81687) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:54:39.820923 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578c36c-c562-4121-89a6-fc025835c86c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:39.822528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578c36c-c562-4121-89a6-fc025835c86c admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:39.834694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1279/2552] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:54:39.838905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b56ab-3f44-4f2c-a814-e7f45db4fdaa None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:39.848217 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b56ab-3f44-4f2c-a814-e7f45db4fdaa None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:39.852815 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86934ad-bdfd-4fcb-a4fa-3981bbd6c3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:39.853402 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86934ad-bdfd-4fcb-a4fa-3981bbd6c3fc admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:39.853678 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86934ad-bdfd-4fcb-a4fa-3981bbd6c3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:39.853948 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86934ad-bdfd-4fcb-a4fa-3981bbd6c3fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:39.854613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86934ad-bdfd-4fcb-a4fa-3981bbd6c3fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:39.856526 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86934ad-bdfd-4fcb-a4fa-3981bbd6c3fc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:39.884244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1274/2553] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:54:39 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:54:39.888293 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578c36c-c562-4121-89a6-fc025835c86c admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:39.899019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578c36c-c562-4121-89a6-fc025835c86c admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:39.947987 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:39.955785 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ec6bd-fe7b-49f9-a61f-59ecc7fb132b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:39.956401 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec6bd-fe7b-49f9-a61f-59ecc7fb132b admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:39.957137 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec6bd-fe7b-49f9-a61f-59ecc7fb132b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:39.957617 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec6bd-fe7b-49f9-a61f-59ecc7fb132b admin admin] PATH_INFO: `/v3/users` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:39.958540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ec6bd-fe7b-49f9-a61f-59ecc7fb132b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:39.960754 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ec6bd-fe7b-49f9-a61f-59ecc7fb132b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:39.982060 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1280/2554] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:54:39 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:54:39.986837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86934ad-bdfd-4fcb-a4fa-3981bbd6c3fc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:39.996896 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86934ad-bdfd-4fcb-a4fa-3981bbd6c3fc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:40.002012 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebada43-3593-4634-85cc-68b917c48f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:40.002706 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebada43-3593-4634-85cc-68b917c48f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:40.002979 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebada43-3593-4634-85cc-68b917c48f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:40.003245 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebada43-3593-4634-85cc-68b917c48f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:40.003873 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebada43-3593-4634-85cc-68b917c48f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:40.005435 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebada43-3593-4634-85cc-68b917c48f14 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:40.015082 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1275/2555] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:54:40.019537 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ec6bd-fe7b-49f9-a61f-59ecc7fb132b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:40.029417 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ec6bd-fe7b-49f9-a61f-59ecc7fb132b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:40.034390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd471fec-d2bd-491f-a704-aa66529b8916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:40.034982 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd471fec-d2bd-491f-a704-aa66529b8916 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:40.035260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd471fec-d2bd-491f-a704-aa66529b8916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:40.035529 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd471fec-d2bd-491f-a704-aa66529b8916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:40.036171 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd471fec-d2bd-491f-a704-aa66529b8916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:40.037737 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd471fec-d2bd-491f-a704-aa66529b8916 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:40.048420 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1281/2556] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:54:40.052122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebada43-3593-4634-85cc-68b917c48f14 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:40.061734 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebada43-3593-4634-85cc-68b917c48f14 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:40.066749 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f62ee8-e5b0-48df-a46f-5cedaa489eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:40.067380 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f62ee8-e5b0-48df-a46f-5cedaa489eed admin admin] REQUEST_METHOD: `PUT` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:40.067655 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f62ee8-e5b0-48df-a46f-5cedaa489eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:40.067923 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f62ee8-e5b0-48df-a46f-5cedaa489eed admin admin] PATH_INFO: `/v3/projects/4edcbd3cb54844218d40e0c249917c71/users/688307978dad4b189741c9a52b94a2ab/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:40.083095 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f62ee8-e5b0-48df-a46f-5cedaa489eed admin admin] RBAC: Authorizing `identity:create_grant(project_id=4edcbd3cb54844218d40e0c249917c71, user_id=688307978dad4b189741c9a52b94a2ab, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:40.085703 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f62ee8-e5b0-48df-a46f-5cedaa489eed admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:40.098490 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1276/2557] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:54:40 2022] PUT /identity/v3/projects/4edcbd3cb54844218d40e0c249917c71/users/688307978dad4b189741c9a52b94a2ab/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:54:40.109575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4a316-350d-4cc0-a98f-4a5cdaf1bfce None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:40.109861 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4a316-350d-4cc0-a98f-4a5cdaf1bfce None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:40.110131 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4a316-350d-4cc0-a98f-4a5cdaf1bfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:40.137677 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-bed4a316-350d-4cc0-a98f-4a5cdaf1bfce None None] MFA Rules not processed for user `688307978dad4b189741c9a52b94a2ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:54:40.138906 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bed4a316-350d-4cc0-a98f-4a5cdaf1bfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:40.203209 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1282/2558] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:54:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:54:40.214817 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f62ee8-e5b0-48df-a46f-5cedaa489eed admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:54:40.216494 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-33f62ee8-e5b0-48df-a46f-5cedaa489eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:54:40.294590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f62ee8-e5b0-48df-a46f-5cedaa489eed admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:54:40.298970 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b9d28-5401-4a7c-a5b0-5d64936e9e3a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07f8443d5594f47bc81e9f9dc9ce43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:54:40.299569 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b9d28-5401-4a7c-a5b0-5d64936e9e3a service glance] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:54:40.299849 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b9d28-5401-4a7c-a5b0-5d64936e9e3a service glance] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:54:40.300122 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b9d28-5401-4a7c-a5b0-5d64936e9e3a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:54:40.308289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b9d28-5401-4a7c-a5b0-5d64936e9e3a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:54:40.309710 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b9d28-5401-4a7c-a5b0-5d64936e9e3a service glance] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:54:40.319328 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1277/2559] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:54:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:54:50.200115 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=41,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:54:50.315233 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=69,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:55:02.187658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4a316-350d-4cc0-a98f-4a5cdaf1bfce None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:02.189074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-bed4a316-350d-4cc0-a98f-4a5cdaf1bfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:02.261203 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4a316-350d-4cc0-a98f-4a5cdaf1bfce None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:02.266134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07942314-9cab-4abf-91f0-fcbbeb6a865b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:02.266762 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07942314-9cab-4abf-91f0-fcbbeb6a865b service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:02.267040 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07942314-9cab-4abf-91f0-fcbbeb6a865b service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:02.267309 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07942314-9cab-4abf-91f0-fcbbeb6a865b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:02.269019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-07942314-9cab-4abf-91f0-fcbbeb6a865b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:02.281136 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07942314-9cab-4abf-91f0-fcbbeb6a865b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:02.282709 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07942314-9cab-4abf-91f0-fcbbeb6a865b service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:02.292978 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1283/2560] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:55:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:55:12.288230 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=32 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:55:24.306861 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b9d28-5401-4a7c-a5b0-5d64936e9e3a service glance] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:24.368856 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b9d28-5401-4a7c-a5b0-5d64936e9e3a service glance] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:24.373613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:24.374256 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:24.374534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:24.374802 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] PATH_INFO: `/v3/users/688307978dad4b189741c9a52b94a2ab` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:24.375922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] RBAC: Authorizing `identity:delete_user(user_id=688307978dad4b189741c9a52b94a2ab)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:24.377806 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:24.424689 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '688307978dad4b189741c9a52b94a2ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:24.425335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '688307978dad4b189741c9a52b94a2ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:24.425955 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '688307978dad4b189741c9a52b94a2ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:24.432814 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '688307978dad4b189741c9a52b94a2ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:24.444787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '688307978dad4b189741c9a52b94a2ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:24.457156 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1278/2561] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:55:24 2022] DELETE /identity/v3/users/688307978dad4b189741c9a52b94a2ab => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:55:24.461671 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d344ed-1f5f-41bc-ab61-2bdcef655a03 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:24.462220 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d344ed-1f5f-41bc-ab61-2bdcef655a03 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:24.462327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d344ed-1f5f-41bc-ab61-2bdcef655a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:24.482376 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b9d344ed-1f5f-41bc-ab61-2bdcef655a03 None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:55:24.483596 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b9d344ed-1f5f-41bc-ab61-2bdcef655a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:24.544174 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1284/2562] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:55:24.553781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:24.555227 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:24.612324 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d20a4-ee8f-4763-afd9-67807c5a667f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:24.616135 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e107e9-4449-41f5-bf42-e60921b7097a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:24.616728 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e107e9-4449-41f5-bf42-e60921b7097a service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:24.617006 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e107e9-4449-41f5-bf42-e60921b7097a service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:24.617272 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e107e9-4449-41f5-bf42-e60921b7097a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:24.626313 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e107e9-4449-41f5-bf42-e60921b7097a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:24.627674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e107e9-4449-41f5-bf42-e60921b7097a service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:24.637508 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1279/2563] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:55:24.902579 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d344ed-1f5f-41bc-ab61-2bdcef655a03 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:24.904052 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b9d344ed-1f5f-41bc-ab61-2bdcef655a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:24.917414 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d344ed-1f5f-41bc-ab61-2bdcef655a03 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:24.921966 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:24.922700 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:24.923186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:24.923526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] PATH_INFO: `/v3/projects/4edcbd3cb54844218d40e0c249917c71` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:24.924707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4edcbd3cb54844218d40e0c249917c71)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:24.926681 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:24.973404 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4edcbd3cb54844218d40e0c249917c71', 'request_id': 'req-97d9779a-00f7-45c3-86ff-709b400c69e1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:24.979272 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4edcbd3cb54844218d40e0c249917c71', 'request_id': 'req-97d9779a-00f7-45c3-86ff-709b400c69e1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:24.986752 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4edcbd3cb54844218d40e0c249917c71', 'request_id': 'req-97d9779a-00f7-45c3-86ff-709b400c69e1'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:24.998366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] The token cache is being invalidate because project 4edcbd3cb54844218d40e0c249917c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:55:24.998656 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:24.999292 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-97d9779a-00f7-45c3-86ff-709b400c69e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:25.000760 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1285/2564] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:55:24 2022] DELETE /identity/v3/projects/4edcbd3cb54844218d40e0c249917c71 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:55:25.005567 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b356a-28c0-4a36-8c2a-6a919f9eed6d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:25.005848 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b356a-28c0-4a36-8c2a-6a919f9eed6d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:25.006143 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b356a-28c0-4a36-8c2a-6a919f9eed6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:25.025547 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-570b356a-28c0-4a36-8c2a-6a919f9eed6d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:55:25.026787 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-570b356a-28c0-4a36-8c2a-6a919f9eed6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:25.043102 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:55:25.087159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1280/2565] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:55:25.095213 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65299cad-3aa0-4060-92cf-c0b59f9142dd None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:25.095496 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65299cad-3aa0-4060-92cf-c0b59f9142dd None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:25.095762 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65299cad-3aa0-4060-92cf-c0b59f9142dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:25.115043 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-65299cad-3aa0-4060-92cf-c0b59f9142dd None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:55:25.116215 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-65299cad-3aa0-4060-92cf-c0b59f9142dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:25.125069 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1286/2566] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:55:25.129244 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b356a-28c0-4a36-8c2a-6a919f9eed6d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:25.139479 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b356a-28c0-4a36-8c2a-6a919f9eed6d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:25.143996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7eb4f-d226-4023-a33b-459902f218d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:25.144636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7eb4f-d226-4023-a33b-459902f218d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:25.144863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7eb4f-d226-4023-a33b-459902f218d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:25.145132 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7eb4f-d226-4023-a33b-459902f218d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:25.145804 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7eb4f-d226-4023-a33b-459902f218d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:55:25.146101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7eb4f-d226-4023-a33b-459902f218d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:25.147713 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7eb4f-d226-4023-a33b-459902f218d4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:25.159285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1281/2567] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:55:25.163298 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65299cad-3aa0-4060-92cf-c0b59f9142dd None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:25.172568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65299cad-3aa0-4060-92cf-c0b59f9142dd None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:25.177166 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e29b98-d3de-4baa-a45e-7a5e9ea05bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:25.177747 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e29b98-d3de-4baa-a45e-7a5e9ea05bda admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:25.178023 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e29b98-d3de-4baa-a45e-7a5e9ea05bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:25.178390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e29b98-d3de-4baa-a45e-7a5e9ea05bda admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:25.179011 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e29b98-d3de-4baa-a45e-7a5e9ea05bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:25.180867 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e29b98-d3de-4baa-a45e-7a5e9ea05bda admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:25.208775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1287/2568] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:55:25 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:55:25.213026 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7eb4f-d226-4023-a33b-459902f218d4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:25.223333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7eb4f-d226-4023-a33b-459902f218d4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:25.278809 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513af189-8d12-44d1-be78-6efe5941d99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:25.279406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513af189-8d12-44d1-be78-6efe5941d99f admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:25.279685 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513af189-8d12-44d1-be78-6efe5941d99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:25.279954 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513af189-8d12-44d1-be78-6efe5941d99f admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:25.280839 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513af189-8d12-44d1-be78-6efe5941d99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:25.282769 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513af189-8d12-44d1-be78-6efe5941d99f admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:25.304584 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1282/2569] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:55:25 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:55:25.308424 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e29b98-d3de-4baa-a45e-7a5e9ea05bda admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:25.318673 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e29b98-d3de-4baa-a45e-7a5e9ea05bda admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:25.323368 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e5d28-a10a-4e16-b051-24a4f2649b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:25.323981 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e5d28-a10a-4e16-b051-24a4f2649b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:25.324260 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e5d28-a10a-4e16-b051-24a4f2649b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:25.324528 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e5d28-a10a-4e16-b051-24a4f2649b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:25.325164 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e5d28-a10a-4e16-b051-24a4f2649b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:25.326775 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e5d28-a10a-4e16-b051-24a4f2649b4f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:25.336473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1288/2570] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:55:25.340322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513af189-8d12-44d1-be78-6efe5941d99f admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:25.349971 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513af189-8d12-44d1-be78-6efe5941d99f admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:25.355063 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce6293-2896-4916-a298-4efc7bb082af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:25.355674 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce6293-2896-4916-a298-4efc7bb082af admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:25.355944 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce6293-2896-4916-a298-4efc7bb082af admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:25.356210 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce6293-2896-4916-a298-4efc7bb082af admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:25.356837 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce6293-2896-4916-a298-4efc7bb082af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:25.358432 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce6293-2896-4916-a298-4efc7bb082af admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:25.368635 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1283/2571] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:55:25.373311 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e5d28-a10a-4e16-b051-24a4f2649b4f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:25.383259 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e5d28-a10a-4e16-b051-24a4f2649b4f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:25.388455 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8208913-e11f-4ecc-9689-71c3a514011b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:25.389088 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208913-e11f-4ecc-9689-71c3a514011b admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:25.389366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208913-e11f-4ecc-9689-71c3a514011b admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:25.389635 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208913-e11f-4ecc-9689-71c3a514011b admin admin] PATH_INFO: `/v3/projects/149af749e41446268d236caff0d9df81/users/98bae787139842ec8b052326b9183362/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:25.404125 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8208913-e11f-4ecc-9689-71c3a514011b admin admin] RBAC: Authorizing `identity:create_grant(project_id=149af749e41446268d236caff0d9df81, user_id=98bae787139842ec8b052326b9183362, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:25.406975 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8208913-e11f-4ecc-9689-71c3a514011b admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:25.420489 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1289/2572] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:55:25 2022] PUT /identity/v3/projects/149af749e41446268d236caff0d9df81/users/98bae787139842ec8b052326b9183362/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:55:25.431630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cff200-01fa-4f7b-a092-382ecd308dc7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:25.431912 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cff200-01fa-4f7b-a092-382ecd308dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:25.432178 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cff200-01fa-4f7b-a092-382ecd308dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:25.460214 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-43cff200-01fa-4f7b-a092-382ecd308dc7 None None] MFA Rules not processed for user `98bae787139842ec8b052326b9183362`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:55:25.461411 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-43cff200-01fa-4f7b-a092-382ecd308dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:25.524666 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1284/2573] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:55:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:55:25.536488 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8208913-e11f-4ecc-9689-71c3a514011b admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:25.538263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-d8208913-e11f-4ecc-9689-71c3a514011b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:25.561185 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:55:25.597001 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8208913-e11f-4ecc-9689-71c3a514011b admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:25.600551 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b11133-059d-4099-bff0-a8bed85b27e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07f8443d5594f47bc81e9f9dc9ce43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:25.601140 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b11133-059d-4099-bff0-a8bed85b27e8 service glance] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:25.601413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b11133-059d-4099-bff0-a8bed85b27e8 service glance] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:25.601679 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b11133-059d-4099-bff0-a8bed85b27e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:25.610336 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b11133-059d-4099-bff0-a8bed85b27e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:25.611738 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b11133-059d-4099-bff0-a8bed85b27e8 service glance] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:25.621048 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1290/2574] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:55:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:55:35.521870 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:55:35.616333 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=17 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:55:58.693489 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cff200-01fa-4f7b-a092-382ecd308dc7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:58.755071 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cff200-01fa-4f7b-a092-382ecd308dc7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:58.759480 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:58.760090 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:58.760363 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:58.760625 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] PATH_INFO: `/v3/users/98bae787139842ec8b052326b9183362` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:58.761704 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98bae787139842ec8b052326b9183362)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:58.764243 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:58.809280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98bae787139842ec8b052326b9183362'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:58.810492 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98bae787139842ec8b052326b9183362'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:58.811100 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98bae787139842ec8b052326b9183362'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:58.817980 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98bae787139842ec8b052326b9183362'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:58.830425 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98bae787139842ec8b052326b9183362'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:58.844279 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1285/2575] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:55:58 2022] DELETE /identity/v3/users/98bae787139842ec8b052326b9183362 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:55:58.848864 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec21c1-ff91-402d-9671-3b8396bf4a3b None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:58.849357 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec21c1-ff91-402d-9671-3b8396bf4a3b None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:58.849468 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec21c1-ff91-402d-9671-3b8396bf4a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:58.868190 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-0cec21c1-ff91-402d-9671-3b8396bf4a3b None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:55:58.869377 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0cec21c1-ff91-402d-9671-3b8396bf4a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:58.927360 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1291/2576] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:55:58.937745 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:58.939125 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:58.995629 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f5a70-794a-4ac3-969c-b4c52683e1d9 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:59.000084 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e371794-76e5-4df0-ae76-b0e662cc360f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:59.000676 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371794-76e5-4df0-ae76-b0e662cc360f service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.000950 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371794-76e5-4df0-ae76-b0e662cc360f service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.001218 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e371794-76e5-4df0-ae76-b0e662cc360f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:59.010034 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e371794-76e5-4df0-ae76-b0e662cc360f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:59.011414 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e371794-76e5-4df0-ae76-b0e662cc360f service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:59.021003 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1286/2577] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:55:59.287559 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec21c1-ff91-402d-9671-3b8396bf4a3b None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:59.288976 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-0cec21c1-ff91-402d-9671-3b8396bf4a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:59.302063 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec21c1-ff91-402d-9671-3b8396bf4a3b None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:59.307027 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:59.307761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.308045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.308320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] PATH_INFO: `/v3/projects/149af749e41446268d236caff0d9df81` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:59.309452 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=149af749e41446268d236caff0d9df81)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:59.312463 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:59.360150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149af749e41446268d236caff0d9df81', 'request_id': 'req-8f3c021c-7187-405e-a0cd-717fb8f42a5d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:59.365658 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149af749e41446268d236caff0d9df81', 'request_id': 'req-8f3c021c-7187-405e-a0cd-717fb8f42a5d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:59.373242 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149af749e41446268d236caff0d9df81', 'request_id': 'req-8f3c021c-7187-405e-a0cd-717fb8f42a5d'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:59.384486 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] The token cache is being invalidate because project 149af749e41446268d236caff0d9df81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:55:59.385046 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:59.385761 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-8f3c021c-7187-405e-a0cd-717fb8f42a5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:55:59.387320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1292/2578] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:55:59 2022] DELETE /identity/v3/projects/149af749e41446268d236caff0d9df81 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:55:59.392079 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41aec-6e40-4666-8621-0badaa3f532a None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.392365 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41aec-6e40-4666-8621-0badaa3f532a None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.392662 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab41aec-6e40-4666-8621-0badaa3f532a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:59.413184 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-eab41aec-6e40-4666-8621-0badaa3f532a None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:55:59.414438 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-eab41aec-6e40-4666-8621-0badaa3f532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:59.431545 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:55:59.474748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1287/2579] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:55:59.483199 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4b04c-d52d-48ea-a9a9-6429b1160c7e None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.483481 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4b04c-d52d-48ea-a9a9-6429b1160c7e None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.483752 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4b04c-d52d-48ea-a9a9-6429b1160c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:59.503634 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-44c4b04c-d52d-48ea-a9a9-6429b1160c7e None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:55:59.504829 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-44c4b04c-d52d-48ea-a9a9-6429b1160c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:59.512784 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1293/2580] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:55:59.516996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab41aec-6e40-4666-8621-0badaa3f532a None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:59.526875 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab41aec-6e40-4666-8621-0badaa3f532a None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:59.532538 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae2785-d43a-4b63-b8d2-72882445f1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:59.533081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae2785-d43a-4b63-b8d2-72882445f1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.533357 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae2785-d43a-4b63-b8d2-72882445f1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.533654 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae2785-d43a-4b63-b8d2-72882445f1b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:59.534342 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae2785-d43a-4b63-b8d2-72882445f1b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81688) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:55:59.534645 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae2785-d43a-4b63-b8d2-72882445f1b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:59.536237 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae2785-d43a-4b63-b8d2-72882445f1b4 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:59.548497 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1288/2581] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Oct 24 13:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:55:59.552763 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4b04c-d52d-48ea-a9a9-6429b1160c7e None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:59.562245 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4b04c-d52d-48ea-a9a9-6429b1160c7e None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:59.566921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c9f3d-7815-4479-954a-bc7103f8b63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:59.567511 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c9f3d-7815-4479-954a-bc7103f8b63a admin admin] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.567793 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c9f3d-7815-4479-954a-bc7103f8b63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.568064 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c9f3d-7815-4479-954a-bc7103f8b63a admin admin] PATH_INFO: `/v3/projects` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:59.568689 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c9f3d-7815-4479-954a-bc7103f8b63a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:59.570615 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c9f3d-7815-4479-954a-bc7103f8b63a admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:59.599407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1294/2582] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Oct 24 13:55:59 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:55:59.603446 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae2785-d43a-4b63-b8d2-72882445f1b4 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:59.614001 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae2785-d43a-4b63-b8d2-72882445f1b4 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:59.670305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37de33f-0336-4256-96da-e3026f3a1bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:59.670895 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37de33f-0336-4256-96da-e3026f3a1bec admin admin] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.671181 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37de33f-0336-4256-96da-e3026f3a1bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.671453 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37de33f-0336-4256-96da-e3026f3a1bec admin admin] PATH_INFO: `/v3/users` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:59.672077 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37de33f-0336-4256-96da-e3026f3a1bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:59.673972 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37de33f-0336-4256-96da-e3026f3a1bec admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:59.696027 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1289/2583] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Oct 24 13:55:59 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:55:59.699694 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c9f3d-7815-4479-954a-bc7103f8b63a admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:59.710078 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c9f3d-7815-4479-954a-bc7103f8b63a admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:59.714702 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f8821-0d58-42e4-904d-b97e3ffc9bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:59.715294 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f8821-0d58-42e4-904d-b97e3ffc9bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.715572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f8821-0d58-42e4-904d-b97e3ffc9bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.716354 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f8821-0d58-42e4-904d-b97e3ffc9bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:59.717192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f8821-0d58-42e4-904d-b97e3ffc9bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:59.718818 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f8821-0d58-42e4-904d-b97e3ffc9bf7 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:59.728862 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1295/2584] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:55:59.733440 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37de33f-0336-4256-96da-e3026f3a1bec admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:59.743177 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37de33f-0336-4256-96da-e3026f3a1bec admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:59.747810 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b08f77-e001-498a-9d37-1f432c36f485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:59.748394 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08f77-e001-498a-9d37-1f432c36f485 admin admin] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.748673 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08f77-e001-498a-9d37-1f432c36f485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.748946 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08f77-e001-498a-9d37-1f432c36f485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:59.749576 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b08f77-e001-498a-9d37-1f432c36f485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:59.751182 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b08f77-e001-498a-9d37-1f432c36f485 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:59.761468 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1290/2585] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Oct 24 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:55:59.766275 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f8821-0d58-42e4-904d-b97e3ffc9bf7 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:59.777706 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f8821-0d58-42e4-904d-b97e3ffc9bf7 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:59.783286 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcb307-5cb3-4fa9-8201-6f66ec3b2b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:59.783870 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb307-5cb3-4fa9-8201-6f66ec3b2b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.784147 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb307-5cb3-4fa9-8201-6f66ec3b2b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.784415 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb307-5cb3-4fa9-8201-6f66ec3b2b4e admin admin] PATH_INFO: `/v3/projects/378196ab11f8477c88aa439c75f96228/users/8664552ed8cb475ba0f24007dbaa5213/roles/09a932be0c1b481c8494705d2030fc2a` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:59.799086 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcb307-5cb3-4fa9-8201-6f66ec3b2b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=378196ab11f8477c88aa439c75f96228, user_id=8664552ed8cb475ba0f24007dbaa5213, role_id=09a932be0c1b481c8494705d2030fc2a)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:55:59.801811 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcb307-5cb3-4fa9-8201-6f66ec3b2b4e admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:55:59.813874 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1296/2586] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Oct 24 13:55:59 2022] PUT /identity/v3/projects/378196ab11f8477c88aa439c75f96228/users/8664552ed8cb475ba0f24007dbaa5213/roles/09a932be0c1b481c8494705d2030fc2a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:55:59.825755 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368059-b374-49e7-8e1b-3ecf7160a95d None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.826157 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368059-b374-49e7-8e1b-3ecf7160a95d None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.826462 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368059-b374-49e7-8e1b-3ecf7160a95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:55:59.853838 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-37368059-b374-49e7-8e1b-3ecf7160a95d None None] MFA Rules not processed for user `8664552ed8cb475ba0f24007dbaa5213`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:55:59.855078 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-37368059-b374-49e7-8e1b-3ecf7160a95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:59.919613 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1291/2587] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:55:59.930807 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcb307-5cb3-4fa9-8201-6f66ec3b2b4e admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:55:59.932470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-5fbcb307-5cb3-4fa9-8201-6f66ec3b2b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:55:59.954526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:55:59.992182 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcb307-5cb3-4fa9-8201-6f66ec3b2b4e admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:55:59.995834 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe15ea5-7332-44cd-b98a-6440411eaef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:55:59.996436 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15ea5-7332-44cd-b98a-6440411eaef2 service cinder] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:55:59.996715 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15ea5-7332-44cd-b98a-6440411eaef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:55:59.996986 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15ea5-7332-44cd-b98a-6440411eaef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:56:00.005418 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe15ea5-7332-44cd-b98a-6440411eaef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:56:00.006876 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe15ea5-7332-44cd-b98a-6440411eaef2 service cinder] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:56:00.016288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1297/2588] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:55:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:56:00.042151 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37368059-b374-49e7-8e1b-3ecf7160a95d None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:56:00.061005 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:56:00.106225 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37368059-b374-49e7-8e1b-3ecf7160a95d None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:56:00.110988 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:56:00.111612 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:56:00.111894 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:56:00.112167 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] PATH_INFO: `/v3/users/8664552ed8cb475ba0f24007dbaa5213` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:56:00.113318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8664552ed8cb475ba0f24007dbaa5213)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:56:00.115258 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:56:00.160615 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8664552ed8cb475ba0f24007dbaa5213'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:56:00.161280 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8664552ed8cb475ba0f24007dbaa5213'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:56:00.161995 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8664552ed8cb475ba0f24007dbaa5213'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:56:00.168739 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8664552ed8cb475ba0f24007dbaa5213'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:56:00.180318 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8664552ed8cb475ba0f24007dbaa5213'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:56:00.193305 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1292/2589] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:56:00 2022] DELETE /identity/v3/users/8664552ed8cb475ba0f24007dbaa5213 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:56:00.197931 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae49e73-5a26-49d9-8566-b413177a6f7d None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:56:00.198240 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae49e73-5a26-49d9-8566-b413177a6f7d None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:56:00.198515 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae49e73-5a26-49d9-8566-b413177a6f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:56:00.221178 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-fae49e73-5a26-49d9-8566-b413177a6f7d None None] MFA Rules not processed for user `75330c8a27ec4acf96914b6033d2cd61`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:56:00.222407 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fae49e73-5a26-49d9-8566-b413177a6f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:56:00.280739 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1298/2590] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:56:00.290596 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:56:00.291998 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:56:00.348659 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef3c00-e945-4b80-8b0a-c305f81f56fc admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:56:00.352369 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42283bdc-16bd-4a2e-a548-7c296285e87c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:56:00.352996 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283bdc-16bd-4a2e-a548-7c296285e87c service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:56:00.353276 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283bdc-16bd-4a2e-a548-7c296285e87c service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:56:00.353555 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283bdc-16bd-4a2e-a548-7c296285e87c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:56:00.362725 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42283bdc-16bd-4a2e-a548-7c296285e87c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:56:00.364145 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42283bdc-16bd-4a2e-a548-7c296285e87c service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:56:00.373882 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1293/2591] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:56:00.637390 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae49e73-5a26-49d9-8566-b413177a6f7d None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:56:00.638921 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-fae49e73-5a26-49d9-8566-b413177a6f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:56:00.650625 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae49e73-5a26-49d9-8566-b413177a6f7d None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:56:00.655263 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:56:00.656321 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:56:00.656602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:56:00.656868 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] PATH_INFO: `/v3/projects/378196ab11f8477c88aa439c75f96228` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:56:00.657969 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=378196ab11f8477c88aa439c75f96228)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:56:00.660019 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:56:00.708164 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '378196ab11f8477c88aa439c75f96228', 'request_id': 'req-50a92cad-f474-4040-a288-ffb7dc8a0f0f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:56:00.713692 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '378196ab11f8477c88aa439c75f96228', 'request_id': 'req-50a92cad-f474-4040-a288-ffb7dc8a0f0f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:56:00.721837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '378196ab11f8477c88aa439c75f96228', 'request_id': 'req-50a92cad-f474-4040-a288-ffb7dc8a0f0f'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:56:00.733231 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] The token cache is being invalidate because project 378196ab11f8477c88aa439c75f96228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:56:00.733519 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:56:00.734150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:56:00.735653 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1299/2592] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:56:00 2022] DELETE /identity/v3/projects/378196ab11f8477c88aa439c75f96228 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:56:10.369293 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:56:10.729697 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:57:05.063238 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42283bdc-16bd-4a2e-a548-7c296285e87c service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:57:05.064781 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-42283bdc-16bd-4a2e-a548-7c296285e87c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:05.075824 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42283bdc-16bd-4a2e-a548-7c296285e87c service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:57:05.124581 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d752a-e083-4831-a1ce-264f709852c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:57:05.125171 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d752a-e083-4831-a1ce-264f709852c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:05.125443 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d752a-e083-4831-a1ce-264f709852c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:05.125706 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d752a-e083-4831-a1ce-264f709852c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:05.127411 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-738d752a-e083-4831-a1ce-264f709852c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:05.196111 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d752a-e083-4831-a1ce-264f709852c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:57:05.197540 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d752a-e083-4831-a1ce-264f709852c4 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:57:05.209407 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1294/2593] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:57:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:57:06.648733 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:57:06.650187 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:06.663430 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a92cad-f474-4040-a288-ffb7dc8a0f0f admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:57:06.666890 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea2130-f819-4012-b95a-48167b0b444a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:57:06.667473 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2130-f819-4012-b95a-48167b0b444a service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:06.667751 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2130-f819-4012-b95a-48167b0b444a service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:06.668018 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea2130-f819-4012-b95a-48167b0b444a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:06.669788 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-abea2130-f819-4012-b95a-48167b0b444a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:06.683273 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2130-f819-4012-b95a-48167b0b444a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:57:06.684714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea2130-f819-4012-b95a-48167b0b444a service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:57:06.695630 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1300/2594] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:57:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:57:08.288457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d752a-e083-4831-a1ce-264f709852c4 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:57:08.289858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-738d752a-e083-4831-a1ce-264f709852c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:08.345709 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d752a-e083-4831-a1ce-264f709852c4 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:57:08.349285 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8616c5a-5854-48f6-8c24-cf21c4c3ee4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c8c594699c045a09c835910d894896b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:57:08.349905 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8616c5a-5854-48f6-8c24-cf21c4c3ee4b service cinder] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:08.350209 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8616c5a-5854-48f6-8c24-cf21c4c3ee4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:08.350482 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8616c5a-5854-48f6-8c24-cf21c4c3ee4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:08.352203 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-d8616c5a-5854-48f6-8c24-cf21c4c3ee4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:08.407679 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:57:08.410087 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8616c5a-5854-48f6-8c24-cf21c4c3ee4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:57:08.411922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8616c5a-5854-48f6-8c24-cf21c4c3ee4b service cinder] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:57:08.421061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1295/2595] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:57:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:57:16.690869 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=6 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:57:18.416858 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=2 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:57:18.420288 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19498ac5-e4fa-41a7-a3a5-689f5fb88505 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:18.420727 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19498ac5-e4fa-41a7-a3a5-689f5fb88505 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:18.420912 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19498ac5-e4fa-41a7-a3a5-689f5fb88505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:18.440319 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-19498ac5-e4fa-41a7-a3a5-689f5fb88505 None None] MFA Rules not processed for user `568d031158744101a6bb339e51149ed2`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:57:18.441530 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-19498ac5-e4fa-41a7-a3a5-689f5fb88505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:18.448353 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1301/2596] 10.222.0.20 () {58 vars in 958 bytes} [Mon Oct 24 13:57:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:57:23.660041 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f49c-12b6-404b-a5e0-b5861f70f10c None None] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:23.660595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f49c-12b6-404b-a5e0-b5861f70f10c None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:23.660595 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f49c-12b6-404b-a5e0-b5861f70f10c None None] PATH_INFO: `/` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:23.661977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1296/2597] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:57:23.668572 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2fbd5-d8ad-4e56-9a1c-e49964ab48f8 None None] REQUEST_METHOD: `POST` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:23.669101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2fbd5-d8ad-4e56-9a1c-e49964ab48f8 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:23.669101 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2fbd5-d8ad-4e56-9a1c-e49964ab48f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:23.709254 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.auth.core [None req-b0e2fbd5-d8ad-4e56-9a1c-e49964ab48f8 None None] MFA Rules not processed for user `5b8479c571ec449fbfc306d25fbf7fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81687) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:57:23.710608 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b0e2fbd5-d8ad-4e56-9a1c-e49964ab48f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:23.717159 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1302/2598] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:57:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:57:23.729475 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5f49c-12b6-404b-a5e0-b5861f70f10c None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:57:23.731150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-15f5f49c-12b6-404b-a5e0-b5861f70f10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:23.745260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5f49c-12b6-404b-a5e0-b5861f70f10c None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:57:23.748746 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec125a-531e-44c9-b373-0e10e028d0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:57:23.749335 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec125a-531e-44c9-b373-0e10e028d0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:23.749611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec125a-531e-44c9-b373-0e10e028d0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:23.749935 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec125a-531e-44c9-b373-0e10e028d0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:23.758488 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec125a-531e-44c9-b373-0e10e028d0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:57:23.759897 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec125a-531e-44c9-b373-0e10e028d0d2 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:57:23.771845 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1297/2599] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:57:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:57:23.883340 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7339df-3918-4f3a-98b1-ed0c0a2c0db0 None None] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:23.883707 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7339df-3918-4f3a-98b1-ed0c0a2c0db0 None None] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:23.884045 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7339df-3918-4f3a-98b1-ed0c0a2c0db0 None None] PATH_INFO: `/` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:23.885470 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1303/2600] 10.222.0.20 () {54 vars in 921 bytes} [Mon Oct 24 13:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:57:23.890983 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad093ae-75b9-41f9-8bc5-0edd405ef1a7 None None] REQUEST_METHOD: `POST` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:23.891289 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad093ae-75b9-41f9-8bc5-0edd405ef1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:23.891558 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad093ae-75b9-41f9-8bc5-0edd405ef1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:23.931658 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.auth.core [None req-7ad093ae-75b9-41f9-8bc5-0edd405ef1a7 None None] MFA Rules not processed for user `18bb93669846435a85eb34771cfb09fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81688) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:57:23.932863 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7ad093ae-75b9-41f9-8bc5-0edd405ef1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:23.943101 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1298/2601] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Oct 24 13:57:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:57:23.953549 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7339df-3918-4f3a-98b1-ed0c0a2c0db0 None None] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:57:23.964512 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7339df-3918-4f3a-98b1-ed0c0a2c0db0 None None] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:57:23.967990 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf8f59-557d-46c9-884b-c334b8fdc23e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:57:23.968632 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf8f59-557d-46c9-884b-c334b8fdc23e service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:23.968914 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf8f59-557d-46c9-884b-c334b8fdc23e service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:23.969186 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf8f59-557d-46c9-884b-c334b8fdc23e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:23.979575 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf8f59-557d-46c9-884b-c334b8fdc23e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:57:23.981013 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf8f59-557d-46c9-884b-c334b8fdc23e service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:57:23.992265 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1304/2602] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:57:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:57:33.929434 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=15 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:57:33.987947 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=18 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:57:34.355572 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad093ae-75b9-41f9-8bc5-0edd405ef1a7 None None] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:57:34.357173 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-7ad093ae-75b9-41f9-8bc5-0edd405ef1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:34.378093 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad093ae-75b9-41f9-8bc5-0edd405ef1a7 None None] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:57:34.384166 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ec716-a292-4816-a182-a33daa674765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:57:34.384867 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ec716-a292-4816-a182-a33daa674765 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:34.385282 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ec716-a292-4816-a182-a33daa674765 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:34.385722 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ec716-a292-4816-a182-a33daa674765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:34.388159 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-299ec716-a292-4816-a182-a33daa674765 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:34.405665 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ec716-a292-4816-a182-a33daa674765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:57:34.407260 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ec716-a292-4816-a182-a33daa674765 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:57:34.418779 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1299/2603] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:57:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:57:37.169539 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf8f59-557d-46c9-884b-c334b8fdc23e service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:57:37.182134 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf8f59-557d-46c9-884b-c334b8fdc23e service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:57:37.186640 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb20f2-1e53-4d59-ac3a-8c0b282a7253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:57:37.187240 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb20f2-1e53-4d59-ac3a-8c0b282a7253 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:57:37.187518 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb20f2-1e53-4d59-ac3a-8c0b282a7253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:57:37.187788 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb20f2-1e53-4d59-ac3a-8c0b282a7253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:57:37.189842 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-cabb20f2-1e53-4d59-ac3a-8c0b282a7253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:57:37.256320 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb20f2-1e53-4d59-ac3a-8c0b282a7253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:57:37.257657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb20f2-1e53-4d59-ac3a-8c0b282a7253 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:57:37.267139 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1305/2604] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:57:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:57:44.414251 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=6 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:57:47.262790 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=30 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:58:06.978264 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ec716-a292-4816-a182-a33daa674765 service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:58:06.987664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ec716-a292-4816-a182-a33daa674765 service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:58:06.991487 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2498f3-c2a8-4781-a4c9-213d3df21f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:58:06.992081 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2498f3-c2a8-4781-a4c9-213d3df21f42 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:58:06.992360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2498f3-c2a8-4781-a4c9-213d3df21f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:58:06.992630 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2498f3-c2a8-4781-a4c9-213d3df21f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:58:06.994372 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ad2498f3-c2a8-4781-a4c9-213d3df21f42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:58:07.004983 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2498f3-c2a8-4781-a4c9-213d3df21f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:58:07.006422 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2498f3-c2a8-4781-a4c9-213d3df21f42 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:58:07.014321 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1300/2605] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:58:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:58:17.010695 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=6 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:59:14.446315 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb20f2-1e53-4d59-ac3a-8c0b282a7253 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:14.455024 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb20f2-1e53-4d59-ac3a-8c0b282a7253 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:14.459146 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714372c-7a52-4ef2-b0a0-1db61fc32f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:14.459739 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714372c-7a52-4ef2-b0a0-1db61fc32f97 service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:14.460021 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714372c-7a52-4ef2-b0a0-1db61fc32f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:14.460342 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714372c-7a52-4ef2-b0a0-1db61fc32f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:14.461998 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-b714372c-7a52-4ef2-b0a0-1db61fc32f97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:14.520409 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714372c-7a52-4ef2-b0a0-1db61fc32f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:14.522360 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714372c-7a52-4ef2-b0a0-1db61fc32f97 service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:14.531803 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1306/2606] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:59:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:59:15.397007 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2498f3-c2a8-4781-a4c9-213d3df21f42 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:15.398534 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ad2498f3-c2a8-4781-a4c9-213d3df21f42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:15.417274 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2498f3-c2a8-4781-a4c9-213d3df21f42 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:15.425967 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7961fb-7f03-459d-8c9f-9c6493a86bd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18bb93669846435a85eb34771cfb09fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:15.428013 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7961fb-7f03-459d-8c9f-9c6493a86bd6 service nova] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:15.428799 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7961fb-7f03-459d-8c9f-9c6493a86bd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:15.429332 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7961fb-7f03-459d-8c9f-9c6493a86bd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:15.431910 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ce7961fb-7f03-459d-8c9f-9c6493a86bd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:15.446980 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7961fb-7f03-459d-8c9f-9c6493a86bd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:15.448552 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7961fb-7f03-459d-8c9f-9c6493a86bd6 service nova] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:15.460671 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1301/2607] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:59:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:59:19.279194 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714372c-7a52-4ef2-b0a0-1db61fc32f97 service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:19.289327 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714372c-7a52-4ef2-b0a0-1db61fc32f97 service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:19.292854 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb4651-99d0-420c-94c1-1d9adea0c0ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:19.293446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb4651-99d0-420c-94c1-1d9adea0c0ea service neutron] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:19.293720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb4651-99d0-420c-94c1-1d9adea0c0ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:19.293984 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb4651-99d0-420c-94c1-1d9adea0c0ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:19.295773 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-09eb4651-99d0-420c-94c1-1d9adea0c0ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:19.358500 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb4651-99d0-420c-94c1-1d9adea0c0ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:19.359935 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb4651-99d0-420c-94c1-1d9adea0c0ea service neutron] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:19.369919 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1307/2608] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:59:22.452638 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7961fb-7f03-459d-8c9f-9c6493a86bd6 service nova] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:22.454524 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-ce7961fb-7f03-459d-8c9f-9c6493a86bd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:22.552180 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7961fb-7f03-459d-8c9f-9c6493a86bd6 service nova] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:22.557705 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57d43e-dcf6-4355-98da-419ef08b1e32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:22.558379 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57d43e-dcf6-4355-98da-419ef08b1e32 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:22.558723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57d43e-dcf6-4355-98da-419ef08b1e32 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:22.559061 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57d43e-dcf6-4355-98da-419ef08b1e32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:22.561248 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-eb57d43e-dcf6-4355-98da-419ef08b1e32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:22.575731 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57d43e-dcf6-4355-98da-419ef08b1e32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:22.577646 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57d43e-dcf6-4355-98da-419ef08b1e32 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:22.595374 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1302/2609] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:59:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:59:25.725241 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb4651-99d0-420c-94c1-1d9adea0c0ea service neutron] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:25.738129 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb4651-99d0-420c-94c1-1d9adea0c0ea service neutron] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:25.744208 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b18e45-92dd-4fc1-96df-a3a92040c7b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:25.744832 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b18e45-92dd-4fc1-96df-a3a92040c7b9 service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:25.745113 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b18e45-92dd-4fc1-96df-a3a92040c7b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:25.745388 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b18e45-92dd-4fc1-96df-a3a92040c7b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:25.747192 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a4b18e45-92dd-4fc1-96df-a3a92040c7b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:25.769720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b18e45-92dd-4fc1-96df-a3a92040c7b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:25.769720 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b18e45-92dd-4fc1-96df-a3a92040c7b9 service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:25.775756 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1308/2610] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:59:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:59:25.802254 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57d43e-dcf6-4355-98da-419ef08b1e32 service placement] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:25.804071 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-eb57d43e-dcf6-4355-98da-419ef08b1e32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:25.820155 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57d43e-dcf6-4355-98da-419ef08b1e32 service placement] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:25.824739 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:25.825358 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:25.825636 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:25.825903 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] PATH_INFO: `/v3/users/d480fb668fa446debc095215f4d55928` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:25.829392 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d480fb668fa446debc095215f4d55928)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:25.831420 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:25.877703 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd480fb668fa446debc095215f4d55928'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:25.878400 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd480fb668fa446debc095215f4d55928'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:25.879021 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd480fb668fa446debc095215f4d55928'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:25.886807 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd480fb668fa446debc095215f4d55928'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:25.898917 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd480fb668fa446debc095215f4d55928'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:25.911878 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1303/2611] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:59:25 2022] DELETE /identity/v3/users/d480fb668fa446debc095215f4d55928 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:59:26.095526 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b18e45-92dd-4fc1-96df-a3a92040c7b9 service placement] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:26.107399 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b18e45-92dd-4fc1-96df-a3a92040c7b9 service placement] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:26.169244 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:26.169894 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:26.170151 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:26.170510 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] PATH_INFO: `/v3/projects/fb8cb8c044ad40a09528d3e1248642a2` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:26.171602 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb8cb8c044ad40a09528d3e1248642a2)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:26.173483 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:26.197422 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=100 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:59:26.220764 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb8cb8c044ad40a09528d3e1248642a2', 'request_id': 'req-17e5c06a-7114-42de-83ce-5f41856f8b75'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.227009 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb8cb8c044ad40a09528d3e1248642a2', 'request_id': 'req-17e5c06a-7114-42de-83ce-5f41856f8b75'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.234339 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb8cb8c044ad40a09528d3e1248642a2', 'request_id': 'req-17e5c06a-7114-42de-83ce-5f41856f8b75'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.246378 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] The token cache is being invalidate because project fb8cb8c044ad40a09528d3e1248642a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:59:26.246665 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.247297 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.248786 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1309/2612] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:59:26 2022] DELETE /identity/v3/projects/fb8cb8c044ad40a09528d3e1248642a2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:59:26.252826 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:26.254302 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:26.267236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41969a0-d42d-4b6b-b781-22ed96186023 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:26.312442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:59:26.325240 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:26.325868 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:26.326150 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:26.326452 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] PATH_INFO: `/v3/users/c76b09da40e04e36a993ca4239055708` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:26.327590 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c76b09da40e04e36a993ca4239055708)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:26.329496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:26.376047 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c76b09da40e04e36a993ca4239055708'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.376691 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c76b09da40e04e36a993ca4239055708'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.377298 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c76b09da40e04e36a993ca4239055708'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.384410 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c76b09da40e04e36a993ca4239055708'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.396322 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c76b09da40e04e36a993ca4239055708'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.409172 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1304/2613] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:59:26 2022] DELETE /identity/v3/users/c76b09da40e04e36a993ca4239055708 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:59:26.585149 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:26.586631 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:26.601211 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5c06a-7114-42de-83ce-5f41856f8b75 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:26.658249 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:26.658878 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:26.659150 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:26.659413 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] PATH_INFO: `/v3/projects/f912c154514849c6af15ecb9fecd1907` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:26.660521 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f912c154514849c6af15ecb9fecd1907)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:26.663381 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:26.707622 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f912c154514849c6af15ecb9fecd1907', 'request_id': 'req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.713743 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f912c154514849c6af15ecb9fecd1907', 'request_id': 'req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.720782 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f912c154514849c6af15ecb9fecd1907', 'request_id': 'req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.732718 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] The token cache is being invalidate because project f912c154514849c6af15ecb9fecd1907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81687) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:59:26.733313 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.733657 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:26.735185 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1310/2614] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:59:26 2022] DELETE /identity/v3/projects/f912c154514849c6af15ecb9fecd1907 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:59:36.405719 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:59:36.729566 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:59:51.158120 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:51.159603 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:51.248127 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fe3d2-3dac-4278-be1d-6966d8d426c8 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:51.295843 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fceee1-68a1-4870-bd6b-df3bb3521bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b8479c571ec449fbfc306d25fbf7fa2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:51.296442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fceee1-68a1-4870-bd6b-df3bb3521bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:51.296721 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fceee1-68a1-4870-bd6b-df3bb3521bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:51.296985 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fceee1-68a1-4870-bd6b-df3bb3521bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:51.298741 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-54fceee1-68a1-4870-bd6b-df3bb3521bc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:51.380308 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fceee1-68a1-4870-bd6b-df3bb3521bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:51.381738 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fceee1-68a1-4870-bd6b-df3bb3521bc5 service neutron] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:51.391413 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1305/2615] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 13:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:59:57.557540 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:57.559667 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:57.651369 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76580ca-0d9d-43eb-ac92-8a5e0e1a10ca admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:57.656366 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:57.656939 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:57.657218 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:57.657492 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] PATH_INFO: `/v3/users/568d031158744101a6bb339e51149ed2` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:57.658714 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=568d031158744101a6bb339e51149ed2)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:57.661170 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:57.710791 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '568d031158744101a6bb339e51149ed2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:57.711446 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '568d031158744101a6bb339e51149ed2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:57.725561 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '568d031158744101a6bb339e51149ed2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:57.726453 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '568d031158744101a6bb339e51149ed2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:57.733837 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '568d031158744101a6bb339e51149ed2'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:57.748148 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1311/2616] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:59:57 2022] DELETE /identity/v3/users/568d031158744101a6bb339e51149ed2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:59:57.923055 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fceee1-68a1-4870-bd6b-df3bb3521bc5 service neutron] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:57.948333 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=100 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:59:57.954496 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fceee1-68a1-4870-bd6b-df3bb3521bc5 service neutron] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:58.011571 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:58.012198 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:58.012477 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:58.012747 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] PATH_INFO: `/v3/projects/5db8c3fdabf94498948055239da40ca0` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:58.013850 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db8c3fdabf94498948055239da40ca0)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:58.015775 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:58.062359 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db8c3fdabf94498948055239da40ca0', 'request_id': 'req-65e950fd-fe9a-49e5-9478-d8a27515a5ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.069406 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db8c3fdabf94498948055239da40ca0', 'request_id': 'req-65e950fd-fe9a-49e5-9478-d8a27515a5ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.075491 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db8c3fdabf94498948055239da40ca0', 'request_id': 'req-65e950fd-fe9a-49e5-9478-d8a27515a5ab'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.086048 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] The token cache is being invalidate because project 5db8c3fdabf94498948055239da40ca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:59:58.086360 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.086987 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.088448 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1306/2617] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:59:57 2022] DELETE /identity/v3/projects/5db8c3fdabf94498948055239da40ca0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:59:58.092568 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:58.093964 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:58.107779 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f4ee9-cfd0-4861-8201-717c51eb5109 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:58.164698 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:58.165317 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:58.165590 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:58.165909 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] PATH_INFO: `/v3/users/ed31432d45e348028027d30eb4d47b20` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:58.166996 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed31432d45e348028027d30eb4d47b20)` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:58.168911 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:58.190291 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:59:58.217722 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed31432d45e348028027d30eb4d47b20'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.218399 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed31432d45e348028027d30eb4d47b20'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.231342 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed31432d45e348028027d30eb4d47b20'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.231992 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed31432d45e348028027d30eb4d47b20'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.239031 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.notifications [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed31432d45e348028027d30eb4d47b20'} {{(pid=81687) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.251322 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1312/2618] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Oct 24 13:59:58 2022] DELETE /identity/v3/users/ed31432d45e348028027d30eb4d47b20 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:59:58.425229 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:59:58.427723 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:59:58.442789 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e950fd-fe9a-49e5-9478-d8a27515a5ab admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:59:58.504144 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75330c8a27ec4acf96914b6033d2cd61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73bbcd87686646c2a2c2fdd8f0e7bdd8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:59:58.504922 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:59:58.505198 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:59:58.505465 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] PATH_INFO: `/v3/projects/e90aa35ce1724dd78e026c9048e03751` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:59:58.506586 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e90aa35ce1724dd78e026c9048e03751)` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:59:58.508457 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:59:58.544664 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:59:58.556025 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e90aa35ce1724dd78e026c9048e03751', 'request_id': 'req-16428bd5-fce0-43aa-8b9f-39fe163e9833'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.563236 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e90aa35ce1724dd78e026c9048e03751', 'request_id': 'req-16428bd5-fce0-43aa-8b9f-39fe163e9833'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.569748 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e90aa35ce1724dd78e026c9048e03751', 'request_id': 'req-16428bd5-fce0-43aa-8b9f-39fe163e9833'} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.581326 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] The token cache is being invalidate because project e90aa35ce1724dd78e026c9048e03751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81688) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:59:58.581611 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.582557 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.notifications [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81688) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:59:58.584442 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1307/2619] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Oct 24 13:59:58 2022] DELETE /identity/v3/projects/e90aa35ce1724dd78e026c9048e03751 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 14:00:08.247881 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 14:00:08.577014 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 14:00:09.875503 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] Authenticating user token {{(pid=81687) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 14:00:09.877004 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 14:00:09.967941 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02058e6e-2ce0-4be3-b055-49c00c0265f9 admin admin] Validating token access rules against request {{(pid=81687) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 14:00:10.027074 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3dfe5c-6080-406d-bb1a-e4a6ddb8ff8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81687) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 14:00:10.027672 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dfe5c-6080-406d-bb1a-e4a6ddb8ff8f service placement] REQUEST_METHOD: `GET` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 14:00:10.027950 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dfe5c-6080-406d-bb1a-e4a6ddb8ff8f service placement] SCRIPT_NAME: `/identity` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 14:00:10.028217 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dfe5c-6080-406d-bb1a-e4a6ddb8ff8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81687) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 14:00:10.029955 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.fernet_utils [None req-7a3dfe5c-6080-406d-bb1a-e4a6ddb8ff8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81687) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 14:00:10.041688 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3dfe5c-6080-406d-bb1a-e4a6ddb8ff8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 14:00:10.043214 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3dfe5c-6080-406d-bb1a-e4a6ddb8ff8f service placement] RBAC: Authorization granted {{(pid=81687) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 14:00:10.053767 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: [pid: 81687|app: 0|req: 1313/2620] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 14:00:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 14:00:20.048776 dv-cinder-862490-2-iscsi devstack@keystone.service[81687]: DEBUG dbcounter [-] [81687] Writing DB stats keystone:SELECT=67 {{(pid=81687) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 14:05:11.515599 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] Authenticating user token {{(pid=81688) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 14:05:11.516991 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.fernet_utils [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81688) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 14:05:11.532388 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16428bd5-fce0-43aa-8b9f-39fe163e9833 admin admin] Validating token access rules against request {{(pid=81688) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 14:05:11.537429 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0474bcb-8054-4ab0-9c38-1d9d9fa61964 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0ce1753e22d4952ab5d4348b0cffc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5fd8784d747f495795c413736ac5280c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81688) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 14:05:11.538068 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0474bcb-8054-4ab0-9c38-1d9d9fa61964 service placement] REQUEST_METHOD: `GET` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 14:05:11.538385 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0474bcb-8054-4ab0-9c38-1d9d9fa61964 service placement] SCRIPT_NAME: `/identity` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 14:05:11.538659 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0474bcb-8054-4ab0-9c38-1d9d9fa61964 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81688) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 14:05:11.547977 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0474bcb-8054-4ab0-9c38-1d9d9fa61964 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 14:05:11.549544 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0474bcb-8054-4ab0-9c38-1d9d9fa61964 service placement] RBAC: Authorization granted {{(pid=81688) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 14:05:11.559505 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: [pid: 81688|app: 0|req: 1308/2621] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Oct 24 14:05:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 14:05:21.555250 dv-cinder-862490-2-iscsi devstack@keystone.service[81688]: DEBUG dbcounter [-] [81688] Writing DB stats keystone:SELECT=6 {{(pid=81688) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}